Orbiter History

Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 338866 reports (Page 89 of 6778)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.282196932s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:49:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.282225594s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.290149774s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:48:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.290177756s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............................................XX...........] duration: 2.638113335s trace: https://jaeger.dev.storj.io/trace/73f2db13275e353b FAIL (2026-04-06 15:48:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.638139992s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 3.184277132s trace: https://jaeger.dev.storj.io/trace/282f3f952d3c39b9 FAIL (2026-04-06 15:48:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== se Apr 6 15:40:03.441484386 (647h29m6.457153979s) [upload=621870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.441489531 (647h29m6.457159134s) [upload=621870]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.553469608 (647h29m6.569139222s) [seg_pos={0 0}, upload=621870]: Begin upload segment... Apr 6 15:40:03.553567446 (647h29m6.569237067s) [seg_pos={0 0}, upload=621870]: Done begin upload segment. Apr 6 15:40:03.553628274 (647h29m6.569297901s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555144703 (647h29m6.570814317s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555821247 (647h29m6.571490855s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556341504 (647h29m6.572011118s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556967796 (647h29m6.572637406s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557471121 (647h29m6.573140741s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557987513 (647h29m6.573657125s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558483802 (647h29m6.574153419s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558935721 (647h29m6.574605331s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559406769 (647h29m6.575076379s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559878194 (647h29m6.575547802s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.560284715 (647h29m6.575954329s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.561580886 (647h29m6.577250501s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562168696 (647h29m6.577838309s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562669214 (647h29m6.578338827s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563151475 (647h29m6.578821087s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563587570 (647h29m6.579257184s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.564005547 (647h29m6.579675159s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:04.141500404 (647h29m7.157170027s) [upload=621870]: Next returned nil segment Apr 6 15:40:04.141522569 (647h29m7.15719219s) [upload=621870]: Waiting for error group managing segments... Apr 6 15:40:04.304476951 (647h29m7.320146558s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.321029537 (647h29m7.336699148s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.342498739 (647h29m7.358168358s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.694622833 (647h29m7.710292453s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809275676 (647h29m7.824945279s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809632477 (647h29m7.825302116s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.814979498 (647h29m7.830649115s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.820205102 (647h29m7.835874715s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.849152629 (647h29m7.864822251s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.879775032 (647h29m7.895444647s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.133558986 (647h29m8.149228589s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.154919931 (647h29m8.170589541s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:07.991118972 (647h29m11.006788579s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:41:03.128365133 (647h30m6.144034801s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn, address:45.91.116.8:4064): protocol: read tcp 139.59.144.22:55076->45.91.116.8:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128409784 (647h30m6.144079403s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 139.59.144.22:39470->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128581967 (647h30m6.144251626s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:12VJf4TrDJ7So1W3XuGbfnbKMWC8ftHReLgqfWGqjpRWAbwCjrp, address:45.91.116.11:4146): protocol: read tcp 139.59.144.22:53902->45.91.116.11:4146: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128928873 (647h30m6.144598492s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1Hb2ay2TWn5B9Z6UwGTSkJhpGvodkz293N9T6cxSmeCYsZG3Cj, address:45.91.116.10:4104): protocol: read tcp 139.59.144.22:52722->45.91.116.10:4104: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129037650 (647h30m6.144707259s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1BvWxeWEnE4X8CJA6KJKryewizvzdGnbmthSZmr2Rfb7LdtUHV, address:45.91.116.7:4041): protocol: read tcp 139.59.144.22:53046->45.91.116.7:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129088475 (647h30m6.144758077s) [seg_pos={0 0}, upload=621870]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.235548245 (647h30m6.25121786s) [upload=621870]: Done uploading segments... Apr 6 15:41:03.235598454 (647h30m6.251268075s) [upload=621870]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.184305987s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.387930821s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:48:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.38796497s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...............................................XX...........] duration: 1.674392317s trace: https://jaeger.dev.storj.io/trace/3a14434996c4db35 FAIL (2026-04-06 15:48:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.674421283s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................................................XX..........] duration: 2.769934552s trace: https://jaeger.dev.storj.io/trace/73f2db13275e353b FAIL (2026-04-06 15:47:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.769972414s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 3.172592444s trace: https://jaeger.dev.storj.io/trace/282f3f952d3c39b9 FAIL (2026-04-06 15:47:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== se Apr 6 15:40:03.441484386 (647h29m6.457153979s) [upload=621870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.441489531 (647h29m6.457159134s) [upload=621870]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.553469608 (647h29m6.569139222s) [seg_pos={0 0}, upload=621870]: Begin upload segment... Apr 6 15:40:03.553567446 (647h29m6.569237067s) [seg_pos={0 0}, upload=621870]: Done begin upload segment. Apr 6 15:40:03.553628274 (647h29m6.569297901s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555144703 (647h29m6.570814317s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555821247 (647h29m6.571490855s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556341504 (647h29m6.572011118s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556967796 (647h29m6.572637406s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557471121 (647h29m6.573140741s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557987513 (647h29m6.573657125s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558483802 (647h29m6.574153419s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558935721 (647h29m6.574605331s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559406769 (647h29m6.575076379s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559878194 (647h29m6.575547802s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.560284715 (647h29m6.575954329s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.561580886 (647h29m6.577250501s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562168696 (647h29m6.577838309s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562669214 (647h29m6.578338827s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563151475 (647h29m6.578821087s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563587570 (647h29m6.579257184s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.564005547 (647h29m6.579675159s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:04.141500404 (647h29m7.157170027s) [upload=621870]: Next returned nil segment Apr 6 15:40:04.141522569 (647h29m7.15719219s) [upload=621870]: Waiting for error group managing segments... Apr 6 15:40:04.304476951 (647h29m7.320146558s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.321029537 (647h29m7.336699148s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.342498739 (647h29m7.358168358s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.694622833 (647h29m7.710292453s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809275676 (647h29m7.824945279s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809632477 (647h29m7.825302116s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.814979498 (647h29m7.830649115s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.820205102 (647h29m7.835874715s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.849152629 (647h29m7.864822251s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.879775032 (647h29m7.895444647s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.133558986 (647h29m8.149228589s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.154919931 (647h29m8.170589541s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:07.991118972 (647h29m11.006788579s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:41:03.128365133 (647h30m6.144034801s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn, address:45.91.116.8:4064): protocol: read tcp 139.59.144.22:55076->45.91.116.8:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128409784 (647h30m6.144079403s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 139.59.144.22:39470->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128581967 (647h30m6.144251626s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:12VJf4TrDJ7So1W3XuGbfnbKMWC8ftHReLgqfWGqjpRWAbwCjrp, address:45.91.116.11:4146): protocol: read tcp 139.59.144.22:53902->45.91.116.11:4146: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128928873 (647h30m6.144598492s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1Hb2ay2TWn5B9Z6UwGTSkJhpGvodkz293N9T6cxSmeCYsZG3Cj, address:45.91.116.10:4104): protocol: read tcp 139.59.144.22:52722->45.91.116.10:4104: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129037650 (647h30m6.144707259s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1BvWxeWEnE4X8CJA6KJKryewizvzdGnbmthSZmr2Rfb7LdtUHV, address:45.91.116.7:4041): protocol: read tcp 139.59.144.22:53046->45.91.116.7:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129088475 (647h30m6.144758077s) [seg_pos={0 0}, upload=621870]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.235548245 (647h30m6.25121786s) [upload=621870]: Done uploading segments... Apr 6 15:41:03.235598454 (647h30m6.251268075s) [upload=621870]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.172679672s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................................................XX.........] duration: 2.877378751s trace: https://jaeger.dev.storj.io/trace/73f2db13275e353b FAIL (2026-04-06 15:47:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.877434759s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [................................................XX..........] duration: 1.703378952s trace: https://jaeger.dev.storj.io/trace/3a14434996c4db35 FAIL (2026-04-06 15:47:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.703459859s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.393175518s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:47:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.393213951s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 3.991976479s trace: https://jaeger.dev.storj.io/trace/282f3f952d3c39b9 FAIL (2026-04-06 15:47:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== se Apr 6 15:40:03.441484386 (647h29m6.457153979s) [upload=621870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.441489531 (647h29m6.457159134s) [upload=621870]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.553469608 (647h29m6.569139222s) [seg_pos={0 0}, upload=621870]: Begin upload segment... Apr 6 15:40:03.553567446 (647h29m6.569237067s) [seg_pos={0 0}, upload=621870]: Done begin upload segment. Apr 6 15:40:03.553628274 (647h29m6.569297901s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555144703 (647h29m6.570814317s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555821247 (647h29m6.571490855s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556341504 (647h29m6.572011118s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556967796 (647h29m6.572637406s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557471121 (647h29m6.573140741s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557987513 (647h29m6.573657125s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558483802 (647h29m6.574153419s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558935721 (647h29m6.574605331s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559406769 (647h29m6.575076379s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559878194 (647h29m6.575547802s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.560284715 (647h29m6.575954329s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.561580886 (647h29m6.577250501s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562168696 (647h29m6.577838309s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562669214 (647h29m6.578338827s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563151475 (647h29m6.578821087s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563587570 (647h29m6.579257184s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.564005547 (647h29m6.579675159s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:04.141500404 (647h29m7.157170027s) [upload=621870]: Next returned nil segment Apr 6 15:40:04.141522569 (647h29m7.15719219s) [upload=621870]: Waiting for error group managing segments... Apr 6 15:40:04.304476951 (647h29m7.320146558s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.321029537 (647h29m7.336699148s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.342498739 (647h29m7.358168358s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.694622833 (647h29m7.710292453s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809275676 (647h29m7.824945279s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809632477 (647h29m7.825302116s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.814979498 (647h29m7.830649115s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.820205102 (647h29m7.835874715s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.849152629 (647h29m7.864822251s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.879775032 (647h29m7.895444647s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.133558986 (647h29m8.149228589s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.154919931 (647h29m8.170589541s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:07.991118972 (647h29m11.006788579s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:41:03.128365133 (647h30m6.144034801s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn, address:45.91.116.8:4064): protocol: read tcp 139.59.144.22:55076->45.91.116.8:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128409784 (647h30m6.144079403s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 139.59.144.22:39470->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128581967 (647h30m6.144251626s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:12VJf4TrDJ7So1W3XuGbfnbKMWC8ftHReLgqfWGqjpRWAbwCjrp, address:45.91.116.11:4146): protocol: read tcp 139.59.144.22:53902->45.91.116.11:4146: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128928873 (647h30m6.144598492s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1Hb2ay2TWn5B9Z6UwGTSkJhpGvodkz293N9T6cxSmeCYsZG3Cj, address:45.91.116.10:4104): protocol: read tcp 139.59.144.22:52722->45.91.116.10:4104: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129037650 (647h30m6.144707259s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1BvWxeWEnE4X8CJA6KJKryewizvzdGnbmthSZmr2Rfb7LdtUHV, address:45.91.116.7:4041): protocol: read tcp 139.59.144.22:53046->45.91.116.7:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129088475 (647h30m6.144758077s) [seg_pos={0 0}, upload=621870]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.235548245 (647h30m6.25121786s) [upload=621870]: Done uploading segments... Apr 6 15:41:03.235598454 (647h30m6.251268075s) [upload=621870]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.992010783s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.................................................XX.........] duration: 1.591223896s trace: https://jaeger.dev.storj.io/trace/3a14434996c4db35 FAIL (2026-04-06 15:46:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.591253987s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.438977393s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:46:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.439074325s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................................................XX........] duration: 2.91725069s trace: https://jaeger.dev.storj.io/trace/73f2db13275e353b FAIL (2026-04-06 15:46:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.917276651s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..................................................XX........] duration: 1.666709969s trace: https://jaeger.dev.storj.io/trace/3a14434996c4db35 FAIL (2026-04-06 15:46:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.666737841s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.317117237s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:46:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.317167649s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 3.235177258s trace: https://jaeger.dev.storj.io/trace/282f3f952d3c39b9 FAIL (2026-04-06 15:45:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== se Apr 6 15:40:03.441484386 (647h29m6.457153979s) [upload=621870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.441489531 (647h29m6.457159134s) [upload=621870]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.553469608 (647h29m6.569139222s) [seg_pos={0 0}, upload=621870]: Begin upload segment... Apr 6 15:40:03.553567446 (647h29m6.569237067s) [seg_pos={0 0}, upload=621870]: Done begin upload segment. Apr 6 15:40:03.553628274 (647h29m6.569297901s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555144703 (647h29m6.570814317s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555821247 (647h29m6.571490855s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556341504 (647h29m6.572011118s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556967796 (647h29m6.572637406s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557471121 (647h29m6.573140741s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557987513 (647h29m6.573657125s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558483802 (647h29m6.574153419s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558935721 (647h29m6.574605331s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559406769 (647h29m6.575076379s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559878194 (647h29m6.575547802s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.560284715 (647h29m6.575954329s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.561580886 (647h29m6.577250501s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562168696 (647h29m6.577838309s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562669214 (647h29m6.578338827s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563151475 (647h29m6.578821087s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563587570 (647h29m6.579257184s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.564005547 (647h29m6.579675159s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:04.141500404 (647h29m7.157170027s) [upload=621870]: Next returned nil segment Apr 6 15:40:04.141522569 (647h29m7.15719219s) [upload=621870]: Waiting for error group managing segments... Apr 6 15:40:04.304476951 (647h29m7.320146558s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.321029537 (647h29m7.336699148s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.342498739 (647h29m7.358168358s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.694622833 (647h29m7.710292453s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809275676 (647h29m7.824945279s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809632477 (647h29m7.825302116s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.814979498 (647h29m7.830649115s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.820205102 (647h29m7.835874715s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.849152629 (647h29m7.864822251s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.879775032 (647h29m7.895444647s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.133558986 (647h29m8.149228589s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.154919931 (647h29m8.170589541s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:07.991118972 (647h29m11.006788579s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:41:03.128365133 (647h30m6.144034801s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn, address:45.91.116.8:4064): protocol: read tcp 139.59.144.22:55076->45.91.116.8:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128409784 (647h30m6.144079403s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 139.59.144.22:39470->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128581967 (647h30m6.144251626s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:12VJf4TrDJ7So1W3XuGbfnbKMWC8ftHReLgqfWGqjpRWAbwCjrp, address:45.91.116.11:4146): protocol: read tcp 139.59.144.22:53902->45.91.116.11:4146: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128928873 (647h30m6.144598492s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1Hb2ay2TWn5B9Z6UwGTSkJhpGvodkz293N9T6cxSmeCYsZG3Cj, address:45.91.116.10:4104): protocol: read tcp 139.59.144.22:52722->45.91.116.10:4104: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129037650 (647h30m6.144707259s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1BvWxeWEnE4X8CJA6KJKryewizvzdGnbmthSZmr2Rfb7LdtUHV, address:45.91.116.7:4041): protocol: read tcp 139.59.144.22:53046->45.91.116.7:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129088475 (647h30m6.144758077s) [seg_pos={0 0}, upload=621870]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.235548245 (647h30m6.25121786s) [upload=621870]: Done uploading segments... Apr 6 15:41:03.235598454 (647h30m6.251268075s) [upload=621870]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.235207984s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.323062353s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:45:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.323134085s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................................................XX.......] duration: 2.935511435s trace: https://jaeger.dev.storj.io/trace/73f2db13275e353b FAIL (2026-04-06 15:45:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.935551587s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.350412502s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:45:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.350448148s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...................................................XX.......] duration: 1.566313022s trace: https://jaeger.dev.storj.io/trace/3a14434996c4db35 FAIL (2026-04-06 15:45:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.566349463s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [....................................................XX......] duration: 1.740045489s trace: https://jaeger.dev.storj.io/trace/3a14434996c4db35 FAIL (2026-04-06 15:45:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.740086433s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................................................XX......] duration: 2.674045344s trace: https://jaeger.dev.storj.io/trace/73f2db13275e353b FAIL (2026-04-06 15:44:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.67407411s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....................................................XX.....] duration: 1.573106884s trace: https://jaeger.dev.storj.io/trace/3a14434996c4db35 FAIL (2026-04-06 15:44:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.573178735s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 3.385993643s trace: https://jaeger.dev.storj.io/trace/282f3f952d3c39b9 FAIL (2026-04-06 15:44:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== se Apr 6 15:40:03.441484386 (647h29m6.457153979s) [upload=621870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.441489531 (647h29m6.457159134s) [upload=621870]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.553469608 (647h29m6.569139222s) [seg_pos={0 0}, upload=621870]: Begin upload segment... Apr 6 15:40:03.553567446 (647h29m6.569237067s) [seg_pos={0 0}, upload=621870]: Done begin upload segment. Apr 6 15:40:03.553628274 (647h29m6.569297901s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555144703 (647h29m6.570814317s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555821247 (647h29m6.571490855s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556341504 (647h29m6.572011118s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556967796 (647h29m6.572637406s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557471121 (647h29m6.573140741s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557987513 (647h29m6.573657125s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558483802 (647h29m6.574153419s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558935721 (647h29m6.574605331s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559406769 (647h29m6.575076379s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559878194 (647h29m6.575547802s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.560284715 (647h29m6.575954329s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.561580886 (647h29m6.577250501s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562168696 (647h29m6.577838309s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562669214 (647h29m6.578338827s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563151475 (647h29m6.578821087s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563587570 (647h29m6.579257184s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.564005547 (647h29m6.579675159s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:04.141500404 (647h29m7.157170027s) [upload=621870]: Next returned nil segment Apr 6 15:40:04.141522569 (647h29m7.15719219s) [upload=621870]: Waiting for error group managing segments... Apr 6 15:40:04.304476951 (647h29m7.320146558s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.321029537 (647h29m7.336699148s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.342498739 (647h29m7.358168358s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.694622833 (647h29m7.710292453s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809275676 (647h29m7.824945279s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809632477 (647h29m7.825302116s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.814979498 (647h29m7.830649115s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.820205102 (647h29m7.835874715s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.849152629 (647h29m7.864822251s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.879775032 (647h29m7.895444647s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.133558986 (647h29m8.149228589s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.154919931 (647h29m8.170589541s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:07.991118972 (647h29m11.006788579s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:41:03.128365133 (647h30m6.144034801s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn, address:45.91.116.8:4064): protocol: read tcp 139.59.144.22:55076->45.91.116.8:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128409784 (647h30m6.144079403s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 139.59.144.22:39470->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128581967 (647h30m6.144251626s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:12VJf4TrDJ7So1W3XuGbfnbKMWC8ftHReLgqfWGqjpRWAbwCjrp, address:45.91.116.11:4146): protocol: read tcp 139.59.144.22:53902->45.91.116.11:4146: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128928873 (647h30m6.144598492s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1Hb2ay2TWn5B9Z6UwGTSkJhpGvodkz293N9T6cxSmeCYsZG3Cj, address:45.91.116.10:4104): protocol: read tcp 139.59.144.22:52722->45.91.116.10:4104: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129037650 (647h30m6.144707259s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1BvWxeWEnE4X8CJA6KJKryewizvzdGnbmthSZmr2Rfb7LdtUHV, address:45.91.116.7:4041): protocol: read tcp 139.59.144.22:53046->45.91.116.7:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129088475 (647h30m6.144758077s) [seg_pos={0 0}, upload=621870]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.235548245 (647h30m6.25121786s) [upload=621870]: Done uploading segments... Apr 6 15:41:03.235598454 (647h30m6.251268075s) [upload=621870]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.386044291s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.378375641s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:44:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.378404024s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......................................................XX....] duration: 1.53218219s trace: https://jaeger.dev.storj.io/trace/3a14434996c4db35 FAIL (2026-04-06 15:44:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.532399128s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.304744557s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:44:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.304774511s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 3.331410959s trace: https://jaeger.dev.storj.io/trace/282f3f952d3c39b9 FAIL (2026-04-06 15:44:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== se Apr 6 15:40:03.441484386 (647h29m6.457153979s) [upload=621870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.441489531 (647h29m6.457159134s) [upload=621870]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.553469608 (647h29m6.569139222s) [seg_pos={0 0}, upload=621870]: Begin upload segment... Apr 6 15:40:03.553567446 (647h29m6.569237067s) [seg_pos={0 0}, upload=621870]: Done begin upload segment. Apr 6 15:40:03.553628274 (647h29m6.569297901s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555144703 (647h29m6.570814317s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555821247 (647h29m6.571490855s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556341504 (647h29m6.572011118s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556967796 (647h29m6.572637406s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557471121 (647h29m6.573140741s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557987513 (647h29m6.573657125s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558483802 (647h29m6.574153419s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558935721 (647h29m6.574605331s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559406769 (647h29m6.575076379s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559878194 (647h29m6.575547802s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.560284715 (647h29m6.575954329s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.561580886 (647h29m6.577250501s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562168696 (647h29m6.577838309s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562669214 (647h29m6.578338827s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563151475 (647h29m6.578821087s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563587570 (647h29m6.579257184s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.564005547 (647h29m6.579675159s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:04.141500404 (647h29m7.157170027s) [upload=621870]: Next returned nil segment Apr 6 15:40:04.141522569 (647h29m7.15719219s) [upload=621870]: Waiting for error group managing segments... Apr 6 15:40:04.304476951 (647h29m7.320146558s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.321029537 (647h29m7.336699148s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.342498739 (647h29m7.358168358s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.694622833 (647h29m7.710292453s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809275676 (647h29m7.824945279s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809632477 (647h29m7.825302116s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.814979498 (647h29m7.830649115s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.820205102 (647h29m7.835874715s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.849152629 (647h29m7.864822251s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.879775032 (647h29m7.895444647s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.133558986 (647h29m8.149228589s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.154919931 (647h29m8.170589541s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:07.991118972 (647h29m11.006788579s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:41:03.128365133 (647h30m6.144034801s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn, address:45.91.116.8:4064): protocol: read tcp 139.59.144.22:55076->45.91.116.8:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128409784 (647h30m6.144079403s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 139.59.144.22:39470->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128581967 (647h30m6.144251626s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:12VJf4TrDJ7So1W3XuGbfnbKMWC8ftHReLgqfWGqjpRWAbwCjrp, address:45.91.116.11:4146): protocol: read tcp 139.59.144.22:53902->45.91.116.11:4146: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128928873 (647h30m6.144598492s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1Hb2ay2TWn5B9Z6UwGTSkJhpGvodkz293N9T6cxSmeCYsZG3Cj, address:45.91.116.10:4104): protocol: read tcp 139.59.144.22:52722->45.91.116.10:4104: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129037650 (647h30m6.144707259s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1BvWxeWEnE4X8CJA6KJKryewizvzdGnbmthSZmr2Rfb7LdtUHV, address:45.91.116.7:4041): protocol: read tcp 139.59.144.22:53046->45.91.116.7:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129088475 (647h30m6.144758077s) [seg_pos={0 0}, upload=621870]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.235548245 (647h30m6.25121786s) [upload=621870]: Done uploading segments... Apr 6 15:41:03.235598454 (647h30m6.251268075s) [upload=621870]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.331469805s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration: 2.792015095s trace: https://jaeger.dev.storj.io/trace/73f2db13275e353b FAIL (2026-04-06 15:44:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.792050308s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......................................................XX...] duration: 1.798721396s trace: https://jaeger.dev.storj.io/trace/3a14434996c4db35 FAIL (2026-04-06 15:43:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.798755689s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.331834552s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:43:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.331867216s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 3.178048454s trace: https://jaeger.dev.storj.io/trace/282f3f952d3c39b9 FAIL (2026-04-06 15:43:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== se Apr 6 15:40:03.441484386 (647h29m6.457153979s) [upload=621870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.441489531 (647h29m6.457159134s) [upload=621870]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.553469608 (647h29m6.569139222s) [seg_pos={0 0}, upload=621870]: Begin upload segment... Apr 6 15:40:03.553567446 (647h29m6.569237067s) [seg_pos={0 0}, upload=621870]: Done begin upload segment. Apr 6 15:40:03.553628274 (647h29m6.569297901s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555144703 (647h29m6.570814317s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555821247 (647h29m6.571490855s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556341504 (647h29m6.572011118s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556967796 (647h29m6.572637406s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557471121 (647h29m6.573140741s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557987513 (647h29m6.573657125s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558483802 (647h29m6.574153419s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558935721 (647h29m6.574605331s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559406769 (647h29m6.575076379s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559878194 (647h29m6.575547802s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.560284715 (647h29m6.575954329s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.561580886 (647h29m6.577250501s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562168696 (647h29m6.577838309s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562669214 (647h29m6.578338827s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563151475 (647h29m6.578821087s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563587570 (647h29m6.579257184s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.564005547 (647h29m6.579675159s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:04.141500404 (647h29m7.157170027s) [upload=621870]: Next returned nil segment Apr 6 15:40:04.141522569 (647h29m7.15719219s) [upload=621870]: Waiting for error group managing segments... Apr 6 15:40:04.304476951 (647h29m7.320146558s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.321029537 (647h29m7.336699148s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.342498739 (647h29m7.358168358s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.694622833 (647h29m7.710292453s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809275676 (647h29m7.824945279s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809632477 (647h29m7.825302116s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.814979498 (647h29m7.830649115s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.820205102 (647h29m7.835874715s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.849152629 (647h29m7.864822251s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.879775032 (647h29m7.895444647s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.133558986 (647h29m8.149228589s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.154919931 (647h29m8.170589541s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:07.991118972 (647h29m11.006788579s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:41:03.128365133 (647h30m6.144034801s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn, address:45.91.116.8:4064): protocol: read tcp 139.59.144.22:55076->45.91.116.8:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128409784 (647h30m6.144079403s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 139.59.144.22:39470->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128581967 (647h30m6.144251626s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:12VJf4TrDJ7So1W3XuGbfnbKMWC8ftHReLgqfWGqjpRWAbwCjrp, address:45.91.116.11:4146): protocol: read tcp 139.59.144.22:53902->45.91.116.11:4146: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128928873 (647h30m6.144598492s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1Hb2ay2TWn5B9Z6UwGTSkJhpGvodkz293N9T6cxSmeCYsZG3Cj, address:45.91.116.10:4104): protocol: read tcp 139.59.144.22:52722->45.91.116.10:4104: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129037650 (647h30m6.144707259s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1BvWxeWEnE4X8CJA6KJKryewizvzdGnbmthSZmr2Rfb7LdtUHV, address:45.91.116.7:4041): protocol: read tcp 139.59.144.22:53046->45.91.116.7:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129088475 (647h30m6.144758077s) [seg_pos={0 0}, upload=621870]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.235548245 (647h30m6.25121786s) [upload=621870]: Done uploading segments... Apr 6 15:41:03.235598454 (647h30m6.251268075s) [upload=621870]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.178142313s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................................................XX....] duration: 3.210134388s trace: https://jaeger.dev.storj.io/trace/73f2db13275e353b FAIL (2026-04-06 15:43:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.210464869s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 3.213830041s trace: https://jaeger.dev.storj.io/trace/282f3f952d3c39b9 FAIL (2026-04-06 15:43:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== se Apr 6 15:40:03.441484386 (647h29m6.457153979s) [upload=621870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.441489531 (647h29m6.457159134s) [upload=621870]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.553469608 (647h29m6.569139222s) [seg_pos={0 0}, upload=621870]: Begin upload segment... Apr 6 15:40:03.553567446 (647h29m6.569237067s) [seg_pos={0 0}, upload=621870]: Done begin upload segment. Apr 6 15:40:03.553628274 (647h29m6.569297901s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555144703 (647h29m6.570814317s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555821247 (647h29m6.571490855s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556341504 (647h29m6.572011118s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556967796 (647h29m6.572637406s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557471121 (647h29m6.573140741s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557987513 (647h29m6.573657125s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558483802 (647h29m6.574153419s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558935721 (647h29m6.574605331s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559406769 (647h29m6.575076379s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559878194 (647h29m6.575547802s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.560284715 (647h29m6.575954329s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.561580886 (647h29m6.577250501s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562168696 (647h29m6.577838309s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562669214 (647h29m6.578338827s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563151475 (647h29m6.578821087s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563587570 (647h29m6.579257184s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.564005547 (647h29m6.579675159s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:04.141500404 (647h29m7.157170027s) [upload=621870]: Next returned nil segment Apr 6 15:40:04.141522569 (647h29m7.15719219s) [upload=621870]: Waiting for error group managing segments... Apr 6 15:40:04.304476951 (647h29m7.320146558s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.321029537 (647h29m7.336699148s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.342498739 (647h29m7.358168358s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.694622833 (647h29m7.710292453s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809275676 (647h29m7.824945279s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809632477 (647h29m7.825302116s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.814979498 (647h29m7.830649115s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.820205102 (647h29m7.835874715s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.849152629 (647h29m7.864822251s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.879775032 (647h29m7.895444647s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.133558986 (647h29m8.149228589s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.154919931 (647h29m8.170589541s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:07.991118972 (647h29m11.006788579s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:41:03.128365133 (647h30m6.144034801s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn, address:45.91.116.8:4064): protocol: read tcp 139.59.144.22:55076->45.91.116.8:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128409784 (647h30m6.144079403s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 139.59.144.22:39470->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128581967 (647h30m6.144251626s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:12VJf4TrDJ7So1W3XuGbfnbKMWC8ftHReLgqfWGqjpRWAbwCjrp, address:45.91.116.11:4146): protocol: read tcp 139.59.144.22:53902->45.91.116.11:4146: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128928873 (647h30m6.144598492s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1Hb2ay2TWn5B9Z6UwGTSkJhpGvodkz293N9T6cxSmeCYsZG3Cj, address:45.91.116.10:4104): protocol: read tcp 139.59.144.22:52722->45.91.116.10:4104: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129037650 (647h30m6.144707259s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1BvWxeWEnE4X8CJA6KJKryewizvzdGnbmthSZmr2Rfb7LdtUHV, address:45.91.116.7:4041): protocol: read tcp 139.59.144.22:53046->45.91.116.7:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129088475 (647h30m6.144758077s) [seg_pos={0 0}, upload=621870]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.235548245 (647h30m6.25121786s) [upload=621870]: Done uploading segments... Apr 6 15:41:03.235598454 (647h30m6.251268075s) [upload=621870]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.213863118s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........................................................XX..] duration: 1.636449823s trace: https://jaeger.dev.storj.io/trace/3a14434996c4db35 FAIL (2026-04-06 15:43:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.636500628s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................XX...] duration: 2.831217296s trace: https://jaeger.dev.storj.io/trace/73f2db13275e353b FAIL (2026-04-06 15:43:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.831250202s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.359174499s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:42:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.359201154s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 3.254380105s trace: https://jaeger.dev.storj.io/trace/282f3f952d3c39b9 FAIL (2026-04-06 15:42:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== se Apr 6 15:40:03.441484386 (647h29m6.457153979s) [upload=621870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.441489531 (647h29m6.457159134s) [upload=621870]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.553469608 (647h29m6.569139222s) [seg_pos={0 0}, upload=621870]: Begin upload segment... Apr 6 15:40:03.553567446 (647h29m6.569237067s) [seg_pos={0 0}, upload=621870]: Done begin upload segment. Apr 6 15:40:03.553628274 (647h29m6.569297901s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555144703 (647h29m6.570814317s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555821247 (647h29m6.571490855s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556341504 (647h29m6.572011118s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556967796 (647h29m6.572637406s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557471121 (647h29m6.573140741s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557987513 (647h29m6.573657125s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558483802 (647h29m6.574153419s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558935721 (647h29m6.574605331s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559406769 (647h29m6.575076379s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559878194 (647h29m6.575547802s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.560284715 (647h29m6.575954329s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.561580886 (647h29m6.577250501s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562168696 (647h29m6.577838309s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562669214 (647h29m6.578338827s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563151475 (647h29m6.578821087s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563587570 (647h29m6.579257184s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.564005547 (647h29m6.579675159s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:04.141500404 (647h29m7.157170027s) [upload=621870]: Next returned nil segment Apr 6 15:40:04.141522569 (647h29m7.15719219s) [upload=621870]: Waiting for error group managing segments... Apr 6 15:40:04.304476951 (647h29m7.320146558s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.321029537 (647h29m7.336699148s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.342498739 (647h29m7.358168358s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.694622833 (647h29m7.710292453s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809275676 (647h29m7.824945279s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809632477 (647h29m7.825302116s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.814979498 (647h29m7.830649115s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.820205102 (647h29m7.835874715s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.849152629 (647h29m7.864822251s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.879775032 (647h29m7.895444647s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.133558986 (647h29m8.149228589s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.154919931 (647h29m8.170589541s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:07.991118972 (647h29m11.006788579s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:41:03.128365133 (647h30m6.144034801s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn, address:45.91.116.8:4064): protocol: read tcp 139.59.144.22:55076->45.91.116.8:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128409784 (647h30m6.144079403s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 139.59.144.22:39470->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128581967 (647h30m6.144251626s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:12VJf4TrDJ7So1W3XuGbfnbKMWC8ftHReLgqfWGqjpRWAbwCjrp, address:45.91.116.11:4146): protocol: read tcp 139.59.144.22:53902->45.91.116.11:4146: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128928873 (647h30m6.144598492s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1Hb2ay2TWn5B9Z6UwGTSkJhpGvodkz293N9T6cxSmeCYsZG3Cj, address:45.91.116.10:4104): protocol: read tcp 139.59.144.22:52722->45.91.116.10:4104: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129037650 (647h30m6.144707259s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1BvWxeWEnE4X8CJA6KJKryewizvzdGnbmthSZmr2Rfb7LdtUHV, address:45.91.116.7:4041): protocol: read tcp 139.59.144.22:53046->45.91.116.7:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129088475 (647h30m6.144758077s) [seg_pos={0 0}, upload=621870]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.235548245 (647h30m6.25121786s) [upload=621870]: Done uploading segments... Apr 6 15:41:03.235598454 (647h30m6.251268075s) [upload=621870]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.254414309s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.561161155s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:42:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.561195558s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 3.294117315s trace: https://jaeger.dev.storj.io/trace/73f2db13275e353b FAIL (2026-04-06 15:42:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.294169963s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........................................................XX.] duration: 3.705453578s trace: https://jaeger.dev.storj.io/trace/3a14434996c4db35 FAIL (2026-04-06 15:41:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.705481416s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.776005208s trace: https://jaeger.dev.storj.io/trace/282f3f952d3c39b9 FAIL (2026-04-06 15:41:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== se Apr 6 15:40:03.441484386 (647h29m6.457153979s) [upload=621870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.441489531 (647h29m6.457159134s) [upload=621870]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.553469608 (647h29m6.569139222s) [seg_pos={0 0}, upload=621870]: Begin upload segment... Apr 6 15:40:03.553567446 (647h29m6.569237067s) [seg_pos={0 0}, upload=621870]: Done begin upload segment. Apr 6 15:40:03.553628274 (647h29m6.569297901s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555144703 (647h29m6.570814317s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555821247 (647h29m6.571490855s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556341504 (647h29m6.572011118s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556967796 (647h29m6.572637406s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557471121 (647h29m6.573140741s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557987513 (647h29m6.573657125s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558483802 (647h29m6.574153419s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558935721 (647h29m6.574605331s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559406769 (647h29m6.575076379s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559878194 (647h29m6.575547802s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.560284715 (647h29m6.575954329s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.561580886 (647h29m6.577250501s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562168696 (647h29m6.577838309s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562669214 (647h29m6.578338827s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563151475 (647h29m6.578821087s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563587570 (647h29m6.579257184s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.564005547 (647h29m6.579675159s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:04.141500404 (647h29m7.157170027s) [upload=621870]: Next returned nil segment Apr 6 15:40:04.141522569 (647h29m7.15719219s) [upload=621870]: Waiting for error group managing segments... Apr 6 15:40:04.304476951 (647h29m7.320146558s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.321029537 (647h29m7.336699148s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.342498739 (647h29m7.358168358s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.694622833 (647h29m7.710292453s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809275676 (647h29m7.824945279s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809632477 (647h29m7.825302116s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.814979498 (647h29m7.830649115s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.820205102 (647h29m7.835874715s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.849152629 (647h29m7.864822251s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.879775032 (647h29m7.895444647s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.133558986 (647h29m8.149228589s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.154919931 (647h29m8.170589541s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:07.991118972 (647h29m11.006788579s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:41:03.128365133 (647h30m6.144034801s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn, address:45.91.116.8:4064): protocol: read tcp 139.59.144.22:55076->45.91.116.8:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128409784 (647h30m6.144079403s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 139.59.144.22:39470->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128581967 (647h30m6.144251626s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:12VJf4TrDJ7So1W3XuGbfnbKMWC8ftHReLgqfWGqjpRWAbwCjrp, address:45.91.116.11:4146): protocol: read tcp 139.59.144.22:53902->45.91.116.11:4146: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128928873 (647h30m6.144598492s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1Hb2ay2TWn5B9Z6UwGTSkJhpGvodkz293N9T6cxSmeCYsZG3Cj, address:45.91.116.10:4104): protocol: read tcp 139.59.144.22:52722->45.91.116.10:4104: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129037650 (647h30m6.144707259s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1BvWxeWEnE4X8CJA6KJKryewizvzdGnbmthSZmr2Rfb7LdtUHV, address:45.91.116.7:4041): protocol: read tcp 139.59.144.22:53046->45.91.116.7:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129088475 (647h30m6.144758077s) [seg_pos={0 0}, upload=621870]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.235548245 (647h30m6.25121786s) [upload=621870]: Done uploading segments... Apr 6 15:41:03.235598454 (647h30m6.251268075s) [upload=621870]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.776028805s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 35.699864735s trace: https://jaeger.dev.storj.io/trace/282f3f952d3c39b9 FAIL (2026-04-06 15:41:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== se Apr 6 15:40:03.441484386 (647h29m6.457153979s) [upload=621870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.441489531 (647h29m6.457159134s) [upload=621870]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.553469608 (647h29m6.569139222s) [seg_pos={0 0}, upload=621870]: Begin upload segment... Apr 6 15:40:03.553567446 (647h29m6.569237067s) [seg_pos={0 0}, upload=621870]: Done begin upload segment. Apr 6 15:40:03.553628274 (647h29m6.569297901s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555144703 (647h29m6.570814317s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555821247 (647h29m6.571490855s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556341504 (647h29m6.572011118s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556967796 (647h29m6.572637406s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557471121 (647h29m6.573140741s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557987513 (647h29m6.573657125s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558483802 (647h29m6.574153419s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558935721 (647h29m6.574605331s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559406769 (647h29m6.575076379s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559878194 (647h29m6.575547802s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.560284715 (647h29m6.575954329s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.561580886 (647h29m6.577250501s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562168696 (647h29m6.577838309s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562669214 (647h29m6.578338827s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563151475 (647h29m6.578821087s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563587570 (647h29m6.579257184s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.564005547 (647h29m6.579675159s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:04.141500404 (647h29m7.157170027s) [upload=621870]: Next returned nil segment Apr 6 15:40:04.141522569 (647h29m7.15719219s) [upload=621870]: Waiting for error group managing segments... Apr 6 15:40:04.304476951 (647h29m7.320146558s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.321029537 (647h29m7.336699148s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.342498739 (647h29m7.358168358s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.694622833 (647h29m7.710292453s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809275676 (647h29m7.824945279s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809632477 (647h29m7.825302116s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.814979498 (647h29m7.830649115s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.820205102 (647h29m7.835874715s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.849152629 (647h29m7.864822251s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.879775032 (647h29m7.895444647s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.133558986 (647h29m8.149228589s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.154919931 (647h29m8.170589541s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:07.991118972 (647h29m11.006788579s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:41:03.128365133 (647h30m6.144034801s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn, address:45.91.116.8:4064): protocol: read tcp 139.59.144.22:55076->45.91.116.8:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128409784 (647h30m6.144079403s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 139.59.144.22:39470->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128581967 (647h30m6.144251626s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:12VJf4TrDJ7So1W3XuGbfnbKMWC8ftHReLgqfWGqjpRWAbwCjrp, address:45.91.116.11:4146): protocol: read tcp 139.59.144.22:53902->45.91.116.11:4146: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128928873 (647h30m6.144598492s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1Hb2ay2TWn5B9Z6UwGTSkJhpGvodkz293N9T6cxSmeCYsZG3Cj, address:45.91.116.10:4104): protocol: read tcp 139.59.144.22:52722->45.91.116.10:4104: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129037650 (647h30m6.144707259s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1BvWxeWEnE4X8CJA6KJKryewizvzdGnbmthSZmr2Rfb7LdtUHV, address:45.91.116.7:4041): protocol: read tcp 139.59.144.22:53046->45.91.116.7:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129088475 (647h30m6.144758077s) [seg_pos={0 0}, upload=621870]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.235548245 (647h30m6.25121786s) [upload=621870]: Done uploading segments... Apr 6 15:41:03.235598454 (647h30m6.251268075s) [upload=621870]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 35.69993256s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 3.192599697s trace: https://jaeger.dev.storj.io/trace/73f2db13275e353b FAIL (2026-04-06 15:41:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.192616683s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.00156538s trace: https://jaeger.dev.storj.io/trace/73f2db13275e353b FAIL (2026-04-06 15:41:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001589084s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.116361343s trace: https://jaeger.dev.storj.io/trace/24b7cbe3868ea029 FAIL (2026-04-06 15:41:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 6 15:40:59.113200131 (647h30m2.128869732s) [upload=621886]: Starting upload Apr 6 15:40:59.113290641 (647h30m2.128960241s) [upload=621886]: Done starting upload Apr 6 15:40:59.113337138 (647h30m2.129006737s) [upload=621886]: Uploading segments... Apr 6 15:40:59.113714459 (647h30m2.12938406s) [upload=621886]: Got next segment. Inline: false Apr 6 15:40:59.113752306 (647h30m2.129421905s) [upload=621886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:59.113757955 (647h30m2.129427556s) [upload=621886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:59.227937557 (647h30m2.243607167s) [seg_pos={0 0}, upload=621886]: Begin upload segment... Apr 6 15:40:59.228033538 (647h30m2.243703143s) [seg_pos={0 0}, upload=621886]: Done begin upload segment. Apr 6 15:40:59.228103967 (647h30m2.243773578s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.228262422 (647h30m2.243932032s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229476673 (647h30m2.245146282s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.229718113 (647h30m2.245387723s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230365564 (647h30m2.246035175s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.230949202 (647h30m2.246618812s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.231495476 (647h30m2.247165086s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232048158 (647h30m2.247717768s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.232612357 (647h30m2.248281973s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.233171134 (647h30m2.248840746s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.234615191 (647h30m2.250284801s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.235641719 (647h30m2.25131133s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236130271 (647h30m2.251799878s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.236592379 (647h30m2.252261989s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237129334 (647h30m2.252798943s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.237618331 (647h30m2.253287942s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Uploading piece... Apr 6 15:40:59.848643716 (647h30m2.864313305s) [upload=621886]: Next returned nil segment Apr 6 15:40:59.848666658 (647h30m2.86433627s) [upload=621886]: Waiting for error group managing segments... Apr 6 15:40:59.974661025 (647h30m2.990330629s) [noise=true , address=145.79.227.8:4335 , piece_id=GMEW5VVZU2T7A7JJHD55Y42BELN7GHELYKDS5HMXCZ2Z2ODYXKNQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.977168292 (647h30m2.992837897s) [noise=true , address=145.79.227.5:4221 , piece_id=CF7TZGIABSWVW6LMRB56C73YVNG3VNLDDRLIDKHASKR7QUDFZBQA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.980846415 (647h30m2.996516027s) [noise=true , address=145.79.227.9:4369 , piece_id=D6CQLVXPXLKI4T2HYHAMAIQOYCU5ZPGPQX6354XBT76HCOKY4PLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.985649355 (647h30m3.001318961s) [noise=true , address=145.79.227.26:4654 , piece_id=KN2AZUA2FVY2MIGGU2VVBQ3J5DCNDS65P5N5XI2KKAWMDP43CENQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.986739092 (647h30m3.002408692s) [noise=true , address=145.79.227.28:4728 , piece_id=7G2PIBQ63IYHR5QRY3I4P7L3PJD5FGFW7OS3SX43RB2MIMJEHPNA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.990327795 (647h30m3.005997394s) [noise=true , address=145.79.227.10:4407 , piece_id=KMGRPGXUV2NDWTO4AF6GP3C2PU6JRVJCBZTRFWRTQPF4D52M6QRQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995230737 (647h30m3.010900338s) [noise=true , address=77.90.31.10:4521 , piece_id=2GQHJWWWWQXS6ONYPQAADT35SD2VXASHD6QPPORDUQ4U5RRHNXEA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:40:59.995433521 (647h30m3.011103121s) [noise=true , address=145.79.227.49:4341 , piece_id=5U4BA3RNELXP5O2WHRBR3TGIVECARMBDC5TMO6KFTMQRFU2KMAHQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.009137822 (647h30m3.024807425s) [noise=true , address=145.79.227.36:4521 , piece_id=LZ4SXHOK3CPBX4F3MROFGFYU6DK2Q2DT37CQRQJPRWW4ST5QDL4A, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.013755800 (647h30m3.029425421s) [noise=true , address=145.79.227.47:4271 , piece_id=4BTQYDRMTS4HQO47UH7YTZNR6HAD67EAXXX2EAP7VPZBYCLTBIYQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:00.015419724 (647h30m3.031089331s) [noise=true , address=145.79.227.7:4298 , piece_id=W4DFL3YTIFLWXHUGD5AHBFH2KAYD6OXZQRPNIDBUSBONB4FZ3RCA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: <nil> Apr 6 15:41:28.813097401 (647h30m31.828767021s) [noise=true , address=45.91.116.12:4225 , piece_id=LWUL2GB55F7SUQRF3R5RPASCTEVDSFRDJ766D3LEJY2CLHC63ZXA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:49794->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813155500 (647h30m31.82882511s) [noise=true , address=45.91.116.14:4191 , piece_id=NYIKIILPYKFOEWHZVFEPYGDUY5LLY77ONQTBUZSC5IRY36RVZBLA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:41748->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813349159 (647h30m31.829018773s) [noise=true , address=45.91.116.10:4084 , piece_id=AAYLK4EF7YTF5FY24EYUIMUYZDH5ZB323EHGAVEOGGHVW4UF7ESQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:40602->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813525379 (647h30m31.82919501s) [noise=true , address=45.91.116.7:4024 , piece_id=MDVTPODCR2DAMWTJQS5N3WUOI47TOARP2XO6FVNN6S2JRQ7DLWJA, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: read tcp 139.59.144.22:34542->45.91.116.7:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813589353 (647h30m31.82925895s) [noise=true , address=45.91.116.5:4301 , piece_id=IFUTQA2JKHEAAECQUDLOXNYNT3RW2ALZ62ZXHAKT5JQLNDLMFKIQ, seg_pos={0 0}, upload=621886]: Done uploading piece. err: ecclient: upload failed (node:12WdUew8BjuMfHyBJ55HwsNFXuWRnBZeNhkCtdhotTgsjSx2Rpg, address:45.91.116.5:4301): protocol: read tcp 139.59.144.22:59900->45.91.116.5:4301: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:28.813632781 (647h30m31.829302376s) [seg_pos={0 0}, upload=621886]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 6 15:41:28.926471512 (647h30m31.942141122s) [upload=621886]: Done uploading segments... Apr 6 15:41:28.926518632 (647h30m31.942188244s) [upload=621886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.116385328s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........................................................XX] duration: 5.002304883s trace: https://jaeger.dev.storj.io/trace/3a14434996c4db35 FAIL (2026-04-06 15:41:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.00233025s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.109971678s trace: https://jaeger.dev.storj.io/trace/282f3f952d3c39b9 FAIL (2026-04-06 15:41:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== se Apr 6 15:40:03.441484386 (647h29m6.457153979s) [upload=621870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.441489531 (647h29m6.457159134s) [upload=621870]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.553469608 (647h29m6.569139222s) [seg_pos={0 0}, upload=621870]: Begin upload segment... Apr 6 15:40:03.553567446 (647h29m6.569237067s) [seg_pos={0 0}, upload=621870]: Done begin upload segment. Apr 6 15:40:03.553628274 (647h29m6.569297901s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555144703 (647h29m6.570814317s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.555821247 (647h29m6.571490855s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556341504 (647h29m6.572011118s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.556967796 (647h29m6.572637406s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557471121 (647h29m6.573140741s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.557987513 (647h29m6.573657125s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558483802 (647h29m6.574153419s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.558935721 (647h29m6.574605331s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559406769 (647h29m6.575076379s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.559878194 (647h29m6.575547802s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.560284715 (647h29m6.575954329s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.561580886 (647h29m6.577250501s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562168696 (647h29m6.577838309s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.562669214 (647h29m6.578338827s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563151475 (647h29m6.578821087s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.563587570 (647h29m6.579257184s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:03.564005547 (647h29m6.579675159s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Uploading piece... Apr 6 15:40:04.141500404 (647h29m7.157170027s) [upload=621870]: Next returned nil segment Apr 6 15:40:04.141522569 (647h29m7.15719219s) [upload=621870]: Waiting for error group managing segments... Apr 6 15:40:04.304476951 (647h29m7.320146558s) [noise=true , address=77.90.31.15:4725 , piece_id=VI6OSDYYRO46VA5H6DTGNDFMFHV2YR3W6NM6VGVHFVTBVZXVSQQA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.321029537 (647h29m7.336699148s) [noise=true , address=77.90.31.21:4901 , piece_id=4HQ7WVAG5CSNISSNIMTHPGXS6UJZHV3YLGFLVTLST5AT6E3BTWQQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.342498739 (647h29m7.358168358s) [noise=true , address=77.90.31.16:4764 , piece_id=GJCUUOFV3ZJU2AZ2TPOE5MEWAOI3ASIMDXRL5G7NXLF6NSEYVMDA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.694622833 (647h29m7.710292453s) [noise=true , address=167.17.72.70:4768 , piece_id=RDZ63BNZQXF7GMGYX657BDZ4KOHE2AKUU2M2RP3PLQ64TU4Z3IWA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809275676 (647h29m7.824945279s) [noise=true , address=193.149.164.70:4121 , piece_id=BPVV7IOUP5CHD3RLFJSJ42HMX32HQQBYV2KSRKSDYXY2GO2VY43A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.809632477 (647h29m7.825302116s) [noise=true , address=193.149.164.66:4996 , piece_id=FFQNVW5QA7ULTWFYOAZLSRAV3HEZ3GNCZFRDWGCGO7ZDECBXWMBA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.814979498 (647h29m7.830649115s) [noise=true , address=193.149.164.67:4031 , piece_id=UCNEMJJ5BJG7ZB5UO37K65WWJCW6OMTH4ZKWONDE3KZB6EQ6NR3Q, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.820205102 (647h29m7.835874715s) [noise=true , address=193.149.164.68:4042 , piece_id=627ADQ4TYIXZJSBTF4EBJV3VABBQ5B2VJXDVJZKYX7OU5VYDWA4A, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.849152629 (647h29m7.864822251s) [noise=true , address=193.149.164.71:4318 , piece_id=JRCNCOHOUEGR7L2R3FHEVG6MHVY5SFBPLPJTIWIUH2FQ4GISV3CQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:04.879775032 (647h29m7.895444647s) [noise=true , address=193.149.164.69:4096 , piece_id=LWTYRPJD4ZQTLVRQUTF7UZCYEIAHV2VPYM56UFEM6WH4OKFFZTJA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.133558986 (647h29m8.149228589s) [noise=true , address=167.17.72.71:4802 , piece_id=3EFW2RW7IOA456GXAZSHPSTM3GTAEVSBG7PM2QOSDMU373Q4H7KQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:05.154919931 (647h29m8.170589541s) [noise=true , address=145.79.227.34:4919 , piece_id=Q66QF3VVWEYSEPVKTLKUMP6FNCGJ36PDWZTS6DQTHBJDL6APHYUA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:40:07.991118972 (647h29m11.006788579s) [noise=true , address=193.149.164.143:4702 , piece_id=CY32N3QDFHAQQGAZFKYCNTWKRAUOTY6W45D2ARQN35FZYATOA4FA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: <nil> Apr 6 15:41:03.128365133 (647h30m6.144034801s) [noise=true , address=45.91.116.8:4064 , piece_id=X6CA5OICXGE6J3Y3EMVIEQHYD2LENWQT4KQQEX3647WNMJTOJCAQ, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:124guAM82oibYZsMho67Em6Ke6yEZXwT6LPMLLSZZBLqwoFGUkn, address:45.91.116.8:4064): protocol: read tcp 139.59.144.22:55076->45.91.116.8:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128409784 (647h30m6.144079403s) [noise=true , address=45.91.116.12:4238 , piece_id=GOFVGN4YQAASX2SVPKBY2DYQAAYT5CA3WIYTPHVOPAHLWXUW6ACA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 139.59.144.22:39470->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128581967 (647h30m6.144251626s) [noise=true , address=45.91.116.11:4146 , piece_id=AROW6W6PTKV27KOQTL2R5VI3YKVTAFR6RW23IXIKUAFVJXURY2IA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:12VJf4TrDJ7So1W3XuGbfnbKMWC8ftHReLgqfWGqjpRWAbwCjrp, address:45.91.116.11:4146): protocol: read tcp 139.59.144.22:53902->45.91.116.11:4146: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.128928873 (647h30m6.144598492s) [noise=true , address=45.91.116.10:4104 , piece_id=MH5BZRGQLZBTEDTOEMTOTCZ6WOT2IQWULZO5GBPCA43XIUWSK5NA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1Hb2ay2TWn5B9Z6UwGTSkJhpGvodkz293N9T6cxSmeCYsZG3Cj, address:45.91.116.10:4104): protocol: read tcp 139.59.144.22:52722->45.91.116.10:4104: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129037650 (647h30m6.144707259s) [noise=true , address=45.91.116.7:4041 , piece_id=TJ34L3TWFXW2IKGCJH2C4YUWH22AXEAVNG2ZOFEWDCMOBSDPKCFA, seg_pos={0 0}, upload=621870]: Done uploading piece. err: ecclient: upload failed (node:1BvWxeWEnE4X8CJA6KJKryewizvzdGnbmthSZmr2Rfb7LdtUHV, address:45.91.116.7:4041): protocol: read tcp 139.59.144.22:53046->45.91.116.7:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.129088475 (647h30m6.144758077s) [seg_pos={0 0}, upload=621870]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.235548245 (647h30m6.25121786s) [upload=621870]: Done uploading segments... Apr 6 15:41:03.235598454 (647h30m6.251268075s) [upload=621870]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.110013736s core uplink us-select-1 us1-placement-20

1
...
87
88
89
90
91
...
6778