Orbiter History

Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 354467 reports (Page 578 of 7090)

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [.....................................................XXXXXXX] duration: 5.001980304s trace: https://jaeger.dev.storj.io/trace/3b340925d737c46f FAIL (2026-03-18 19:39:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002026584s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................XXX.X] duration: 1m0.000749538s trace: https://jaeger.dev.storj.io/trace/25df35b29a44893a FAIL (2026-03-18 19:38:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000773152s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X..X..] duration: 3.107521096s trace: https://jaeger.dev.storj.io/trace/2ca59d04a6d58ba0 FAIL (2026-03-18 19:38:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=187737]: Uploading segments... Mar 18 19:37:12.981533109 (195h26m15.997202698s) [upload=187737]: Got next segment. Inline: false Mar 18 19:37:12.981598888 (195h26m15.997268489s) [upload=187737]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:37:12.981604720 (195h26m15.997274321s) [upload=187737]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:37:13.094870952 (195h26m16.110540559s) [seg_pos={0 0}, upload=187737]: Begin upload segment... Mar 18 19:37:13.094952447 (195h26m16.110622051s) [seg_pos={0 0}, upload=187737]: Done begin upload segment. Mar 18 19:37:13.095058924 (195h26m16.110728528s) [noise=true , address=77.90.31.19:4859 , piece_id=DSDO54RPJDJL74QLHBW5CWHNMQEUOWQXCPEX2KTA3YJXCJRH7Z2Q, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095069831 (195h26m16.110739444s) [noise=true , address=77.90.31.11:4567 , piece_id=HUJKKLGH32QQAFHNPKPX4Q6ZTOT6PE2U3LGU2YGR3ZO53EOVZZDQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095349501 (195h26m16.111019107s) [noise=true , address=79.127.219.45:4883 , piece_id=7NB4IV6P2KFYO62WORCI3OICZWN2QD4LVTXMP2AAXO2Q5Y65JGKQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095876166 (195h26m16.11154576s) [noise=true , address=193.149.164.71:4298 , piece_id=JJ7AR3N4XQHCDCPGNWRU5VH5UBSBBLBPVLVPGR7UZGFB6WBF2DTA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095979098 (195h26m16.111648712s) [noise=true , address=145.79.227.4:4203 , piece_id=JQ7NCR6S3UVYLZ27OGYRIC75O2EGDRMS7VCZX5SADZAHAKR2IXIA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.096545094 (195h26m16.112214708s) [noise=true , address=193.149.164.68:4052 , piece_id=DJN4FAGSUCIPGE4TP2CH2LANKMAQXLR3JYR3KONIEFWA5S3JGMTA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.097159648 (195h26m16.11282925s) [noise=true , address=45.91.116.12:4237 , piece_id=XYQGCI2ET6MHS3YHU7NX5BAZKS2KNTOWRS5YTH24KPUOJGZMG3EA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.097411902 (195h26m16.11308151s) [noise=true , address=79.127.219.46:4907 , piece_id=F552ZCGYCFVHQAVBL2UMTAR4JVFRX7HHBLUS7ZFL6XAQZVT7UPIQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.098280639 (195h26m16.113950249s) [noise=true , address=145.79.227.17:4977 , piece_id=EGN563W6WKAQ5XEMCVOWG62HDX6LRGDA4SNGFNIINCKS6IGDCYKQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.098832187 (195h26m16.114501796s) [noise=true , address=77.90.31.12:4609 , piece_id=VWCJAFHE5CQV6QKNCSXP4JG3YFWKOCC7LRGQGI3YNNGBLWKOECYQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.099348103 (195h26m16.115017706s) [noise=true , address=145.79.227.38:4986 , piece_id=POHAECJMSBQTPCZSBCUN3OTVKWTYFJHWTJ6KG3TQ4NJXLT4OYDMA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.099809042 (195h26m16.115478648s) [noise=true , address=193.149.164.67:4010 , piece_id=6DIUZIS2OU3C55O52CQDWCAJVRHNYMQN4SXZXDXDYVK2B6LVDXLQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.100280986 (195h26m16.115950599s) [noise=true , address=167.17.72.71:4808 , piece_id=YOSBTFC3H6LTHYDW2I2KGJ3GH4ITNXJF6EUMMIWG3XJ2T4W4IJZA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.101049801 (195h26m16.116719407s) [noise=true , address=193.149.164.66:4977 , piece_id=FYH4GQM4Q66RUWIFGFGI45OCIJGH2XNDJCF4L2N6UPYFCKJJHYFA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.101995144 (195h26m16.117664757s) [noise=true , address=145.79.227.21:4485 , piece_id=HPET4TTOR3KPMYP6UVXMMIRZIVHWHEAXW3KJCLVF7LDEL4PJHBRQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.102105850 (195h26m16.117775458s) [noise=true , address=77.90.31.14:4687 , piece_id=2SNQMF43X42PS4DFJ4A4Q4PJQZ4W2P7V6INFCIFQJ2DOAWP7RV2Q, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.102538807 (195h26m16.118208418s) [noise=true , address=193.149.164.69:4095 , piece_id=OKKGKGL3OPLUUTNAML7KS7Z355OQTQADDU7ZH4VEJGNHDMWE6ZAA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.102937431 (195h26m16.118607039s) [noise=true , address=145.79.227.43:4175 , piece_id=TQRTZBPACZZ3SU4CZ4C2IWS7EOFBL5ES4H3SAHSZXSWOZHTLMJTA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.654721924 (195h26m16.670391534s) [upload=187737]: Next returned nil segment Mar 18 19:37:13.654739810 (195h26m16.670409421s) [upload=187737]: Waiting for error group managing segments... Mar 18 19:37:13.794878973 (195h26m16.810548577s) [noise=true , address=77.90.31.12:4609 , piece_id=VWCJAFHE5CQV6QKNCSXP4JG3YFWKOCC7LRGQGI3YNNGBLWKOECYQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.822282354 (195h26m16.837951958s) [noise=true , address=77.90.31.11:4567 , piece_id=HUJKKLGH32QQAFHNPKPX4Q6ZTOT6PE2U3LGU2YGR3ZO53EOVZZDQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.824574466 (195h26m16.84024407s) [noise=true , address=79.127.219.45:4883 , piece_id=7NB4IV6P2KFYO62WORCI3OICZWN2QD4LVTXMP2AAXO2Q5Y65JGKQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.824736917 (195h26m16.840406527s) [noise=true , address=45.91.116.12:4237 , piece_id=XYQGCI2ET6MHS3YHU7NX5BAZKS2KNTOWRS5YTH24KPUOJGZMG3EA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.836649120 (195h26m16.852318721s) [noise=true , address=79.127.219.46:4907 , piece_id=F552ZCGYCFVHQAVBL2UMTAR4JVFRX7HHBLUS7ZFL6XAQZVT7UPIQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.882465904 (195h26m16.898135505s) [noise=true , address=77.90.31.19:4859 , piece_id=DSDO54RPJDJL74QLHBW5CWHNMQEUOWQXCPEX2KTA3YJXCJRH7Z2Q, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.899738045 (195h26m16.915407652s) [noise=true , address=77.90.31.14:4687 , piece_id=2SNQMF43X42PS4DFJ4A4Q4PJQZ4W2P7V6INFCIFQJ2DOAWP7RV2Q, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.163170463 (195h26m17.178840082s) [noise=true , address=167.17.72.71:4808 , piece_id=YOSBTFC3H6LTHYDW2I2KGJ3GH4ITNXJF6EUMMIWG3XJ2T4W4IJZA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.313991205 (195h26m17.329660806s) [noise=true , address=193.149.164.71:4298 , piece_id=JJ7AR3N4XQHCDCPGNWRU5VH5UBSBBLBPVLVPGR7UZGFB6WBF2DTA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.317475737 (195h26m17.333145341s) [noise=true , address=193.149.164.67:4010 , piece_id=6DIUZIS2OU3C55O52CQDWCAJVRHNYMQN4SXZXDXDYVK2B6LVDXLQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.318408375 (195h26m17.334077977s) [noise=true , address=193.149.164.66:4977 , piece_id=FYH4GQM4Q66RUWIFGFGI45OCIJGH2XNDJCF4L2N6UPYFCKJJHYFA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.321516393 (195h26m17.337185997s) [noise=true , address=193.149.164.69:4095 , piece_id=OKKGKGL3OPLUUTNAML7KS7Z355OQTQADDU7ZH4VEJGNHDMWE6ZAA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.346324910 (195h26m17.361994521s) [noise=true , address=193.149.164.68:4052 , piece_id=DJN4FAGSUCIPGE4TP2CH2LANKMAQXLR3JYR3KONIEFWA5S3JGMTA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:38:12.698035371 (195h27m15.713704979s) [noise=true , address=145.79.227.21:4485 , piece_id=HPET4TTOR3KPMYP6UVXMMIRZIVHWHEAXW3KJCLVF7LDEL4PJHBRQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:1wmohWyFaMkNm8KBAZprmZ6RuiH6UcB8KRNght2HGpV3Ucnnq9, address:145.79.227.21:4485): protocol: rpc: dial tcp 145.79.227.21:4485: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698112463 (195h27m15.713782073s) [noise=true , address=145.79.227.17:4977 , piece_id=EGN563W6WKAQ5XEMCVOWG62HDX6LRGDA4SNGFNIINCKS6IGDCYKQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:145.79.227.17:4977): protocol: rpc: dial tcp 145.79.227.17:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698176988 (195h27m15.713846593s) [noise=true , address=145.79.227.38:4986 , piece_id=POHAECJMSBQTPCZSBCUN3OTVKWTYFJHWTJ6KG3TQ4NJXLT4OYDMA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:1xiyNnuK7RcA3pYJRz2QStzUVfaU6gVUovxX2gV86xAQ2K9np4, address:145.79.227.38:4986): protocol: rpc: dial tcp 145.79.227.38:4986: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698193160 (195h27m15.713862769s) [noise=true , address=145.79.227.43:4175 , piece_id=TQRTZBPACZZ3SU4CZ4C2IWS7EOFBL5ES4H3SAHSZXSWOZHTLMJTA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:145.79.227.43:4175): protocol: rpc: dial tcp 145.79.227.43:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698493447 (195h27m15.714163057s) [noise=true , address=145.79.227.4:4203 , piece_id=JQ7NCR6S3UVYLZ27OGYRIC75O2EGDRMS7VCZX5SADZAHAKR2IXIA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: rpc: dial tcp 145.79.227.4:4203: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698532860 (195h27m15.714202463s) [seg_pos={0 0}, upload=187737]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 18 19:38:12.801009852 (195h27m15.816679453s) [upload=187737]: Done uploading segments... Mar 18 19:38:12.801045568 (195h27m15.81671517s) [upload=187737]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.107552663s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................X..X.] duration: 3.331791904s trace: https://jaeger.dev.storj.io/trace/2ca59d04a6d58ba0 FAIL (2026-03-18 19:38:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=187737]: Uploading segments... Mar 18 19:37:12.981533109 (195h26m15.997202698s) [upload=187737]: Got next segment. Inline: false Mar 18 19:37:12.981598888 (195h26m15.997268489s) [upload=187737]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:37:12.981604720 (195h26m15.997274321s) [upload=187737]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:37:13.094870952 (195h26m16.110540559s) [seg_pos={0 0}, upload=187737]: Begin upload segment... Mar 18 19:37:13.094952447 (195h26m16.110622051s) [seg_pos={0 0}, upload=187737]: Done begin upload segment. Mar 18 19:37:13.095058924 (195h26m16.110728528s) [noise=true , address=77.90.31.19:4859 , piece_id=DSDO54RPJDJL74QLHBW5CWHNMQEUOWQXCPEX2KTA3YJXCJRH7Z2Q, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095069831 (195h26m16.110739444s) [noise=true , address=77.90.31.11:4567 , piece_id=HUJKKLGH32QQAFHNPKPX4Q6ZTOT6PE2U3LGU2YGR3ZO53EOVZZDQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095349501 (195h26m16.111019107s) [noise=true , address=79.127.219.45:4883 , piece_id=7NB4IV6P2KFYO62WORCI3OICZWN2QD4LVTXMP2AAXO2Q5Y65JGKQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095876166 (195h26m16.11154576s) [noise=true , address=193.149.164.71:4298 , piece_id=JJ7AR3N4XQHCDCPGNWRU5VH5UBSBBLBPVLVPGR7UZGFB6WBF2DTA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095979098 (195h26m16.111648712s) [noise=true , address=145.79.227.4:4203 , piece_id=JQ7NCR6S3UVYLZ27OGYRIC75O2EGDRMS7VCZX5SADZAHAKR2IXIA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.096545094 (195h26m16.112214708s) [noise=true , address=193.149.164.68:4052 , piece_id=DJN4FAGSUCIPGE4TP2CH2LANKMAQXLR3JYR3KONIEFWA5S3JGMTA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.097159648 (195h26m16.11282925s) [noise=true , address=45.91.116.12:4237 , piece_id=XYQGCI2ET6MHS3YHU7NX5BAZKS2KNTOWRS5YTH24KPUOJGZMG3EA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.097411902 (195h26m16.11308151s) [noise=true , address=79.127.219.46:4907 , piece_id=F552ZCGYCFVHQAVBL2UMTAR4JVFRX7HHBLUS7ZFL6XAQZVT7UPIQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.098280639 (195h26m16.113950249s) [noise=true , address=145.79.227.17:4977 , piece_id=EGN563W6WKAQ5XEMCVOWG62HDX6LRGDA4SNGFNIINCKS6IGDCYKQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.098832187 (195h26m16.114501796s) [noise=true , address=77.90.31.12:4609 , piece_id=VWCJAFHE5CQV6QKNCSXP4JG3YFWKOCC7LRGQGI3YNNGBLWKOECYQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.099348103 (195h26m16.115017706s) [noise=true , address=145.79.227.38:4986 , piece_id=POHAECJMSBQTPCZSBCUN3OTVKWTYFJHWTJ6KG3TQ4NJXLT4OYDMA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.099809042 (195h26m16.115478648s) [noise=true , address=193.149.164.67:4010 , piece_id=6DIUZIS2OU3C55O52CQDWCAJVRHNYMQN4SXZXDXDYVK2B6LVDXLQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.100280986 (195h26m16.115950599s) [noise=true , address=167.17.72.71:4808 , piece_id=YOSBTFC3H6LTHYDW2I2KGJ3GH4ITNXJF6EUMMIWG3XJ2T4W4IJZA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.101049801 (195h26m16.116719407s) [noise=true , address=193.149.164.66:4977 , piece_id=FYH4GQM4Q66RUWIFGFGI45OCIJGH2XNDJCF4L2N6UPYFCKJJHYFA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.101995144 (195h26m16.117664757s) [noise=true , address=145.79.227.21:4485 , piece_id=HPET4TTOR3KPMYP6UVXMMIRZIVHWHEAXW3KJCLVF7LDEL4PJHBRQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.102105850 (195h26m16.117775458s) [noise=true , address=77.90.31.14:4687 , piece_id=2SNQMF43X42PS4DFJ4A4Q4PJQZ4W2P7V6INFCIFQJ2DOAWP7RV2Q, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.102538807 (195h26m16.118208418s) [noise=true , address=193.149.164.69:4095 , piece_id=OKKGKGL3OPLUUTNAML7KS7Z355OQTQADDU7ZH4VEJGNHDMWE6ZAA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.102937431 (195h26m16.118607039s) [noise=true , address=145.79.227.43:4175 , piece_id=TQRTZBPACZZ3SU4CZ4C2IWS7EOFBL5ES4H3SAHSZXSWOZHTLMJTA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.654721924 (195h26m16.670391534s) [upload=187737]: Next returned nil segment Mar 18 19:37:13.654739810 (195h26m16.670409421s) [upload=187737]: Waiting for error group managing segments... Mar 18 19:37:13.794878973 (195h26m16.810548577s) [noise=true , address=77.90.31.12:4609 , piece_id=VWCJAFHE5CQV6QKNCSXP4JG3YFWKOCC7LRGQGI3YNNGBLWKOECYQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.822282354 (195h26m16.837951958s) [noise=true , address=77.90.31.11:4567 , piece_id=HUJKKLGH32QQAFHNPKPX4Q6ZTOT6PE2U3LGU2YGR3ZO53EOVZZDQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.824574466 (195h26m16.84024407s) [noise=true , address=79.127.219.45:4883 , piece_id=7NB4IV6P2KFYO62WORCI3OICZWN2QD4LVTXMP2AAXO2Q5Y65JGKQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.824736917 (195h26m16.840406527s) [noise=true , address=45.91.116.12:4237 , piece_id=XYQGCI2ET6MHS3YHU7NX5BAZKS2KNTOWRS5YTH24KPUOJGZMG3EA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.836649120 (195h26m16.852318721s) [noise=true , address=79.127.219.46:4907 , piece_id=F552ZCGYCFVHQAVBL2UMTAR4JVFRX7HHBLUS7ZFL6XAQZVT7UPIQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.882465904 (195h26m16.898135505s) [noise=true , address=77.90.31.19:4859 , piece_id=DSDO54RPJDJL74QLHBW5CWHNMQEUOWQXCPEX2KTA3YJXCJRH7Z2Q, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.899738045 (195h26m16.915407652s) [noise=true , address=77.90.31.14:4687 , piece_id=2SNQMF43X42PS4DFJ4A4Q4PJQZ4W2P7V6INFCIFQJ2DOAWP7RV2Q, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.163170463 (195h26m17.178840082s) [noise=true , address=167.17.72.71:4808 , piece_id=YOSBTFC3H6LTHYDW2I2KGJ3GH4ITNXJF6EUMMIWG3XJ2T4W4IJZA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.313991205 (195h26m17.329660806s) [noise=true , address=193.149.164.71:4298 , piece_id=JJ7AR3N4XQHCDCPGNWRU5VH5UBSBBLBPVLVPGR7UZGFB6WBF2DTA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.317475737 (195h26m17.333145341s) [noise=true , address=193.149.164.67:4010 , piece_id=6DIUZIS2OU3C55O52CQDWCAJVRHNYMQN4SXZXDXDYVK2B6LVDXLQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.318408375 (195h26m17.334077977s) [noise=true , address=193.149.164.66:4977 , piece_id=FYH4GQM4Q66RUWIFGFGI45OCIJGH2XNDJCF4L2N6UPYFCKJJHYFA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.321516393 (195h26m17.337185997s) [noise=true , address=193.149.164.69:4095 , piece_id=OKKGKGL3OPLUUTNAML7KS7Z355OQTQADDU7ZH4VEJGNHDMWE6ZAA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.346324910 (195h26m17.361994521s) [noise=true , address=193.149.164.68:4052 , piece_id=DJN4FAGSUCIPGE4TP2CH2LANKMAQXLR3JYR3KONIEFWA5S3JGMTA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:38:12.698035371 (195h27m15.713704979s) [noise=true , address=145.79.227.21:4485 , piece_id=HPET4TTOR3KPMYP6UVXMMIRZIVHWHEAXW3KJCLVF7LDEL4PJHBRQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:1wmohWyFaMkNm8KBAZprmZ6RuiH6UcB8KRNght2HGpV3Ucnnq9, address:145.79.227.21:4485): protocol: rpc: dial tcp 145.79.227.21:4485: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698112463 (195h27m15.713782073s) [noise=true , address=145.79.227.17:4977 , piece_id=EGN563W6WKAQ5XEMCVOWG62HDX6LRGDA4SNGFNIINCKS6IGDCYKQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:145.79.227.17:4977): protocol: rpc: dial tcp 145.79.227.17:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698176988 (195h27m15.713846593s) [noise=true , address=145.79.227.38:4986 , piece_id=POHAECJMSBQTPCZSBCUN3OTVKWTYFJHWTJ6KG3TQ4NJXLT4OYDMA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:1xiyNnuK7RcA3pYJRz2QStzUVfaU6gVUovxX2gV86xAQ2K9np4, address:145.79.227.38:4986): protocol: rpc: dial tcp 145.79.227.38:4986: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698193160 (195h27m15.713862769s) [noise=true , address=145.79.227.43:4175 , piece_id=TQRTZBPACZZ3SU4CZ4C2IWS7EOFBL5ES4H3SAHSZXSWOZHTLMJTA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:145.79.227.43:4175): protocol: rpc: dial tcp 145.79.227.43:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698493447 (195h27m15.714163057s) [noise=true , address=145.79.227.4:4203 , piece_id=JQ7NCR6S3UVYLZ27OGYRIC75O2EGDRMS7VCZX5SADZAHAKR2IXIA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: rpc: dial tcp 145.79.227.4:4203: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698532860 (195h27m15.714202463s) [seg_pos={0 0}, upload=187737]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 18 19:38:12.801009852 (195h27m15.816679453s) [upload=187737]: Done uploading segments... Mar 18 19:38:12.801045568 (195h27m15.81671517s) [upload=187737]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.331814746s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................X..X] duration: 1m0.105382764s trace: https://jaeger.dev.storj.io/trace/2ca59d04a6d58ba0 FAIL (2026-03-18 19:38:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=187737]: Uploading segments... Mar 18 19:37:12.981533109 (195h26m15.997202698s) [upload=187737]: Got next segment. Inline: false Mar 18 19:37:12.981598888 (195h26m15.997268489s) [upload=187737]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:37:12.981604720 (195h26m15.997274321s) [upload=187737]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:37:13.094870952 (195h26m16.110540559s) [seg_pos={0 0}, upload=187737]: Begin upload segment... Mar 18 19:37:13.094952447 (195h26m16.110622051s) [seg_pos={0 0}, upload=187737]: Done begin upload segment. Mar 18 19:37:13.095058924 (195h26m16.110728528s) [noise=true , address=77.90.31.19:4859 , piece_id=DSDO54RPJDJL74QLHBW5CWHNMQEUOWQXCPEX2KTA3YJXCJRH7Z2Q, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095069831 (195h26m16.110739444s) [noise=true , address=77.90.31.11:4567 , piece_id=HUJKKLGH32QQAFHNPKPX4Q6ZTOT6PE2U3LGU2YGR3ZO53EOVZZDQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095349501 (195h26m16.111019107s) [noise=true , address=79.127.219.45:4883 , piece_id=7NB4IV6P2KFYO62WORCI3OICZWN2QD4LVTXMP2AAXO2Q5Y65JGKQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095876166 (195h26m16.11154576s) [noise=true , address=193.149.164.71:4298 , piece_id=JJ7AR3N4XQHCDCPGNWRU5VH5UBSBBLBPVLVPGR7UZGFB6WBF2DTA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095979098 (195h26m16.111648712s) [noise=true , address=145.79.227.4:4203 , piece_id=JQ7NCR6S3UVYLZ27OGYRIC75O2EGDRMS7VCZX5SADZAHAKR2IXIA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.096545094 (195h26m16.112214708s) [noise=true , address=193.149.164.68:4052 , piece_id=DJN4FAGSUCIPGE4TP2CH2LANKMAQXLR3JYR3KONIEFWA5S3JGMTA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.097159648 (195h26m16.11282925s) [noise=true , address=45.91.116.12:4237 , piece_id=XYQGCI2ET6MHS3YHU7NX5BAZKS2KNTOWRS5YTH24KPUOJGZMG3EA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.097411902 (195h26m16.11308151s) [noise=true , address=79.127.219.46:4907 , piece_id=F552ZCGYCFVHQAVBL2UMTAR4JVFRX7HHBLUS7ZFL6XAQZVT7UPIQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.098280639 (195h26m16.113950249s) [noise=true , address=145.79.227.17:4977 , piece_id=EGN563W6WKAQ5XEMCVOWG62HDX6LRGDA4SNGFNIINCKS6IGDCYKQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.098832187 (195h26m16.114501796s) [noise=true , address=77.90.31.12:4609 , piece_id=VWCJAFHE5CQV6QKNCSXP4JG3YFWKOCC7LRGQGI3YNNGBLWKOECYQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.099348103 (195h26m16.115017706s) [noise=true , address=145.79.227.38:4986 , piece_id=POHAECJMSBQTPCZSBCUN3OTVKWTYFJHWTJ6KG3TQ4NJXLT4OYDMA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.099809042 (195h26m16.115478648s) [noise=true , address=193.149.164.67:4010 , piece_id=6DIUZIS2OU3C55O52CQDWCAJVRHNYMQN4SXZXDXDYVK2B6LVDXLQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.100280986 (195h26m16.115950599s) [noise=true , address=167.17.72.71:4808 , piece_id=YOSBTFC3H6LTHYDW2I2KGJ3GH4ITNXJF6EUMMIWG3XJ2T4W4IJZA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.101049801 (195h26m16.116719407s) [noise=true , address=193.149.164.66:4977 , piece_id=FYH4GQM4Q66RUWIFGFGI45OCIJGH2XNDJCF4L2N6UPYFCKJJHYFA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.101995144 (195h26m16.117664757s) [noise=true , address=145.79.227.21:4485 , piece_id=HPET4TTOR3KPMYP6UVXMMIRZIVHWHEAXW3KJCLVF7LDEL4PJHBRQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.102105850 (195h26m16.117775458s) [noise=true , address=77.90.31.14:4687 , piece_id=2SNQMF43X42PS4DFJ4A4Q4PJQZ4W2P7V6INFCIFQJ2DOAWP7RV2Q, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.102538807 (195h26m16.118208418s) [noise=true , address=193.149.164.69:4095 , piece_id=OKKGKGL3OPLUUTNAML7KS7Z355OQTQADDU7ZH4VEJGNHDMWE6ZAA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.102937431 (195h26m16.118607039s) [noise=true , address=145.79.227.43:4175 , piece_id=TQRTZBPACZZ3SU4CZ4C2IWS7EOFBL5ES4H3SAHSZXSWOZHTLMJTA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.654721924 (195h26m16.670391534s) [upload=187737]: Next returned nil segment Mar 18 19:37:13.654739810 (195h26m16.670409421s) [upload=187737]: Waiting for error group managing segments... Mar 18 19:37:13.794878973 (195h26m16.810548577s) [noise=true , address=77.90.31.12:4609 , piece_id=VWCJAFHE5CQV6QKNCSXP4JG3YFWKOCC7LRGQGI3YNNGBLWKOECYQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.822282354 (195h26m16.837951958s) [noise=true , address=77.90.31.11:4567 , piece_id=HUJKKLGH32QQAFHNPKPX4Q6ZTOT6PE2U3LGU2YGR3ZO53EOVZZDQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.824574466 (195h26m16.84024407s) [noise=true , address=79.127.219.45:4883 , piece_id=7NB4IV6P2KFYO62WORCI3OICZWN2QD4LVTXMP2AAXO2Q5Y65JGKQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.824736917 (195h26m16.840406527s) [noise=true , address=45.91.116.12:4237 , piece_id=XYQGCI2ET6MHS3YHU7NX5BAZKS2KNTOWRS5YTH24KPUOJGZMG3EA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.836649120 (195h26m16.852318721s) [noise=true , address=79.127.219.46:4907 , piece_id=F552ZCGYCFVHQAVBL2UMTAR4JVFRX7HHBLUS7ZFL6XAQZVT7UPIQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.882465904 (195h26m16.898135505s) [noise=true , address=77.90.31.19:4859 , piece_id=DSDO54RPJDJL74QLHBW5CWHNMQEUOWQXCPEX2KTA3YJXCJRH7Z2Q, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.899738045 (195h26m16.915407652s) [noise=true , address=77.90.31.14:4687 , piece_id=2SNQMF43X42PS4DFJ4A4Q4PJQZ4W2P7V6INFCIFQJ2DOAWP7RV2Q, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.163170463 (195h26m17.178840082s) [noise=true , address=167.17.72.71:4808 , piece_id=YOSBTFC3H6LTHYDW2I2KGJ3GH4ITNXJF6EUMMIWG3XJ2T4W4IJZA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.313991205 (195h26m17.329660806s) [noise=true , address=193.149.164.71:4298 , piece_id=JJ7AR3N4XQHCDCPGNWRU5VH5UBSBBLBPVLVPGR7UZGFB6WBF2DTA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.317475737 (195h26m17.333145341s) [noise=true , address=193.149.164.67:4010 , piece_id=6DIUZIS2OU3C55O52CQDWCAJVRHNYMQN4SXZXDXDYVK2B6LVDXLQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.318408375 (195h26m17.334077977s) [noise=true , address=193.149.164.66:4977 , piece_id=FYH4GQM4Q66RUWIFGFGI45OCIJGH2XNDJCF4L2N6UPYFCKJJHYFA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.321516393 (195h26m17.337185997s) [noise=true , address=193.149.164.69:4095 , piece_id=OKKGKGL3OPLUUTNAML7KS7Z355OQTQADDU7ZH4VEJGNHDMWE6ZAA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.346324910 (195h26m17.361994521s) [noise=true , address=193.149.164.68:4052 , piece_id=DJN4FAGSUCIPGE4TP2CH2LANKMAQXLR3JYR3KONIEFWA5S3JGMTA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:38:12.698035371 (195h27m15.713704979s) [noise=true , address=145.79.227.21:4485 , piece_id=HPET4TTOR3KPMYP6UVXMMIRZIVHWHEAXW3KJCLVF7LDEL4PJHBRQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:1wmohWyFaMkNm8KBAZprmZ6RuiH6UcB8KRNght2HGpV3Ucnnq9, address:145.79.227.21:4485): protocol: rpc: dial tcp 145.79.227.21:4485: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698112463 (195h27m15.713782073s) [noise=true , address=145.79.227.17:4977 , piece_id=EGN563W6WKAQ5XEMCVOWG62HDX6LRGDA4SNGFNIINCKS6IGDCYKQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:145.79.227.17:4977): protocol: rpc: dial tcp 145.79.227.17:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698176988 (195h27m15.713846593s) [noise=true , address=145.79.227.38:4986 , piece_id=POHAECJMSBQTPCZSBCUN3OTVKWTYFJHWTJ6KG3TQ4NJXLT4OYDMA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:1xiyNnuK7RcA3pYJRz2QStzUVfaU6gVUovxX2gV86xAQ2K9np4, address:145.79.227.38:4986): protocol: rpc: dial tcp 145.79.227.38:4986: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698193160 (195h27m15.713862769s) [noise=true , address=145.79.227.43:4175 , piece_id=TQRTZBPACZZ3SU4CZ4C2IWS7EOFBL5ES4H3SAHSZXSWOZHTLMJTA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:145.79.227.43:4175): protocol: rpc: dial tcp 145.79.227.43:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698493447 (195h27m15.714163057s) [noise=true , address=145.79.227.4:4203 , piece_id=JQ7NCR6S3UVYLZ27OGYRIC75O2EGDRMS7VCZX5SADZAHAKR2IXIA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: rpc: dial tcp 145.79.227.4:4203: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698532860 (195h27m15.714202463s) [seg_pos={0 0}, upload=187737]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 18 19:38:12.801009852 (195h27m15.816679453s) [upload=187737]: Done uploading segments... Mar 18 19:38:12.801045568 (195h27m15.81671517s) [upload=187737]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.105407376s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [......................................................XXXXXX] duration: 5.001718592s trace: https://jaeger.dev.storj.io/trace/2d7533bbf15d50b9 FAIL (2026-03-18 19:38:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001742471s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 30.117237131s trace: https://jaeger.dev.storj.io/trace/6c129bb3387c539d FAIL (2026-03-18 19:37:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ress=77.90.31.15:4701 , piece_id=V56EN62YVC42LANKATXJHQEDDHMM4YEQS5JIGWN3YFR2YZRT6HVA, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.706785508 (195h26m31.722455119s) [noise=true , address=145.79.227.29:4945 , piece_id=EEFZFM5PDQVDWZHN3PCKQ2HVVJOWVRQV2ZPM3XTBATWOE6R3OI7Q, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.707464250 (195h26m31.72313386s) [noise=true , address=145.79.227.30:4764 , piece_id=NHZRRMPRFBJDNL7KSOWJBI5GSVDXJB5SAGNLW4I43V7D6VIBRMQA, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.708021831 (195h26m31.723691433s) [noise=true , address=145.79.227.5:4226 , piece_id=V7KSYTU6LHUODHH5MVSD6KSO4VZCLZKG3NI4L5MPOPDTIRVC2H3Q, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.708643554 (195h26m31.724313164s) [noise=true , address=77.90.31.11:4553 , piece_id=SFOEKV2T23O5653REJM4HTP6FNH23ZW2JFSDRZK4GV6YHGOUYZUQ, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.709346094 (195h26m31.725015706s) [noise=true , address=145.79.227.14:4841 , piece_id=AYUNMF6SSUQ2BNINBNJTDD7WSIQAJ7JAAM3TYRWZUDE426WBTSZA, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.710602127 (195h26m31.726271737s) [noise=true , address=145.79.227.17:4975 , piece_id=25QFZB7VCEQ6A2EUUW4CHYCPBSPJ3SBRYEFGWYST5JAASEPSBHGA, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.711932073 (195h26m31.727601683s) [noise=true , address=145.79.227.52:4446 , piece_id=US53DSEOKIKQIPTAYXCNXWH6TVSGWBALEMK65O5FDKGOW2PCC3NA, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.713153547 (195h26m31.728823163s) [noise=true , address=77.90.31.17:4773 , piece_id=M75VYO33ZQXDEZXUYNAYZBH3IOXXJ55VOC5XOZ2ZEZT5LS7YOCJA, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.715153035 (195h26m31.730822636s) [noise=true , address=45.91.116.7:4019 , piece_id=66BR6QQOUQKRMGHX4C5KEHOUBSZYUO52HWVQHXVAI4J6PUJ7JLWQ, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.717242805 (195h26m31.732912417s) [noise=true , address=145.79.227.9:4370 , piece_id=53ZF3PTKN2R4U7LMFJBRDHENQVUE5NOJ42NX2ZMQBILKCCWBLIMQ, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.719197556 (195h26m31.734867167s) [noise=true , address=145.79.227.4:4187 , piece_id=HRTWIK63U2SMGIJOCPLFEEPRSMWINPRMHRKS5T7OL2EQ6LKEPHWQ, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.720983590 (195h26m31.736653202s) [noise=true , address=145.79.227.40:4049 , piece_id=G6Y7AM6JLZWTTREDIIFOZAADS2JL2QPRXYBPGTL7OBFGU5LP3SGA, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.722685044 (195h26m31.738354655s) [noise=true , address=145.79.227.25:4619 , piece_id=UK4O3GO7B6RAG7OHRKOQ7E7PJKWSTQJHPS2MIYXMTUCTZHKTMFRA, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.723750878 (195h26m31.739420479s) [noise=true , address=145.79.227.51:4409 , piece_id=4HIF336GGYWYXOYGFYXWIKGD6NAFVLGQ6ZSQZKF6SR24IKJMJWEQ, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:28.725469723 (195h26m31.741139327s) [noise=true , address=145.79.227.8:4334 , piece_id=M3GBOJPLSDFVFXZOP53J66IEF6LHQIQQHIHUKQL2ILCK5QP3NYZA, seg_pos={0 0}, upload=187741]: Uploading piece... Mar 18 19:37:29.332646860 (195h26m32.348316454s) [upload=187741]: Next returned nil segment Mar 18 19:37:29.332661830 (195h26m32.34833143s) [upload=187741]: Waiting for error group managing segments... Mar 18 19:37:29.470544207 (195h26m32.486213807s) [noise=true , address=77.90.31.11:4553 , piece_id=SFOEKV2T23O5653REJM4HTP6FNH23ZW2JFSDRZK4GV6YHGOUYZUQ, seg_pos={0 0}, upload=187741]: Done uploading piece. err: <nil> Mar 18 19:37:29.471997230 (195h26m32.487666841s) [noise=true , address=45.91.116.7:4019 , piece_id=66BR6QQOUQKRMGHX4C5KEHOUBSZYUO52HWVQHXVAI4J6PUJ7JLWQ, seg_pos={0 0}, upload=187741]: Done uploading piece. err: <nil> Mar 18 19:37:29.536696130 (195h26m32.552365737s) [noise=true , address=77.90.31.15:4701 , piece_id=V56EN62YVC42LANKATXJHQEDDHMM4YEQS5JIGWN3YFR2YZRT6HVA, seg_pos={0 0}, upload=187741]: Done uploading piece. err: <nil> Mar 18 19:37:29.536903166 (195h26m32.552572774s) [noise=true , address=77.90.31.17:4773 , piece_id=M75VYO33ZQXDEZXUYNAYZBH3IOXXJ55VOC5XOZ2ZEZT5LS7YOCJA, seg_pos={0 0}, upload=187741]: Done uploading piece. err: <nil> Mar 18 19:37:58.315393290 (195h27m1.331062899s) [noise=true , address=145.79.227.8:4334 , piece_id=M3GBOJPLSDFVFXZOP53J66IEF6LHQIQQHIHUKQL2ILCK5QP3NYZA, seg_pos={0 0}, upload=187741]: Done uploading piece. err: ecclient: upload failed (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1, address:145.79.227.8:4334): protocol: rpc: dial tcp 145.79.227.8:4334: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:58.315423016 (195h27m1.331092627s) [noise=true , address=145.79.227.4:4187 , piece_id=HRTWIK63U2SMGIJOCPLFEEPRSMWINPRMHRKS5T7OL2EQ6LKEPHWQ, seg_pos={0 0}, upload=187741]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:58.315817951 (195h27m1.331487561s) [noise=true , address=145.79.227.5:4226 , piece_id=V7KSYTU6LHUODHH5MVSD6KSO4VZCLZKG3NI4L5MPOPDTIRVC2H3Q, seg_pos={0 0}, upload=187741]: Done uploading piece. err: ecclient: upload failed (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt, address:145.79.227.5:4226): protocol: rpc: dial tcp 145.79.227.5:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:58.315944322 (195h27m1.331613935s) [noise=true , address=145.79.227.25:4619 , piece_id=UK4O3GO7B6RAG7OHRKOQ7E7PJKWSTQJHPS2MIYXMTUCTZHKTMFRA, seg_pos={0 0}, upload=187741]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: rpc: dial tcp 145.79.227.25:4619: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:58.316000633 (195h27m1.331670238s) [noise=true , address=145.79.227.14:4841 , piece_id=AYUNMF6SSUQ2BNINBNJTDD7WSIQAJ7JAAM3TYRWZUDE426WBTSZA, seg_pos={0 0}, upload=187741]: Done uploading piece. err: ecclient: upload failed (node:12ndNj9Dy8qQSbHXwVWw31sz19ki1qy6Zgetjvj1A5Q3odkzmDm, address:145.79.227.14:4841): protocol: rpc: dial tcp 145.79.227.14:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:58.316229710 (195h27m1.331899319s) [noise=true , address=145.79.227.29:4945 , piece_id=EEFZFM5PDQVDWZHN3PCKQ2HVVJOWVRQV2ZPM3XTBATWOE6R3OI7Q, seg_pos={0 0}, upload=187741]: Done uploading piece. err: ecclient: upload failed (node:128kFn1ZsqrdV936j55GuKpnHdERmtBxcL4B9bZrnjeXMaLpfsP, address:145.79.227.29:4945): protocol: rpc: dial tcp 145.79.227.29:4945: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:58.316363823 (195h27m1.332033432s) [noise=true , address=145.79.227.30:4764 , piece_id=NHZRRMPRFBJDNL7KSOWJBI5GSVDXJB5SAGNLW4I43V7D6VIBRMQA, seg_pos={0 0}, upload=187741]: Done uploading piece. err: ecclient: upload failed (node:127hafVDbun7FoG4JdcogBmRNT6rhL2QQbLDLstFnx7HCWyiC9N, address:145.79.227.30:4764): protocol: rpc: dial tcp 145.79.227.30:4764: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:58.316374978 (195h27m1.332044587s) [noise=true , address=145.79.227.9:4370 , piece_id=53ZF3PTKN2R4U7LMFJBRDHENQVUE5NOJ42NX2ZMQBILKCCWBLIMQ, seg_pos={0 0}, upload=187741]: Done uploading piece. err: ecclient: upload failed (node:12SXZN7WJvcHys9BRd4Lq8NAZ3MSgHf9ZVv7SgaiJNRosdmFVpm, address:145.79.227.9:4370): protocol: rpc: dial tcp 145.79.227.9:4370: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:58.316422470 (195h27m1.332092075s) [noise=true , address=145.79.227.40:4049 , piece_id=G6Y7AM6JLZWTTREDIIFOZAADS2JL2QPRXYBPGTL7OBFGU5LP3SGA, seg_pos={0 0}, upload=187741]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: rpc: dial tcp 145.79.227.40:4049: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:58.316549562 (195h27m1.332219171s) [noise=true , address=145.79.227.17:4975 , piece_id=25QFZB7VCEQ6A2EUUW4CHYCPBSPJ3SBRYEFGWYST5JAASEPSBHGA, seg_pos={0 0}, upload=187741]: Done uploading piece. err: ecclient: upload failed (node:12WiUZeNCJk4rzjtyWuKJwGcZATaYLU2QxN27MZE7i45zVpu9dM, address:145.79.227.17:4975): protocol: rpc: dial tcp 145.79.227.17:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:58.316588087 (195h27m1.332257699s) [noise=true , address=145.79.227.51:4409 , piece_id=4HIF336GGYWYXOYGFYXWIKGD6NAFVLGQ6ZSQZKF6SR24IKJMJWEQ, seg_pos={0 0}, upload=187741]: Done uploading piece. err: ecclient: upload failed (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, address:145.79.227.51:4409): protocol: rpc: dial tcp 145.79.227.51:4409: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:58.316763550 (195h27m1.332433153s) [noise=true , address=145.79.227.52:4446 , piece_id=US53DSEOKIKQIPTAYXCNXWH6TVSGWBALEMK65O5FDKGOW2PCC3NA, seg_pos={0 0}, upload=187741]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:37:58.316804417 (195h27m1.332474023s) [seg_pos={0 0}, upload=187741]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Mar 18 19:37:58.430821353 (195h27m1.446490943s) [upload=187741]: Done uploading segments... Mar 18 19:37:58.430858389 (195h27m1.446527989s) [upload=187741]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 30.117253365s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 30.109352332s trace: https://jaeger.dev.storj.io/trace/3dff7823292fee87 FAIL (2026-03-18 19:37:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== segments... Mar 18 19:36:58.354959697 (195h26m1.370629298s) [upload=187733]: Got next segment. Inline: false Mar 18 19:36:58.355049525 (195h26m1.370719126s) [upload=187733]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:36:58.355056165 (195h26m1.370725769s) [upload=187733]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:36:58.466296822 (195h26m1.481966432s) [seg_pos={0 0}, upload=187733]: Begin upload segment... Mar 18 19:36:58.466366798 (195h26m1.482036405s) [seg_pos={0 0}, upload=187733]: Done begin upload segment. Mar 18 19:36:58.466410116 (195h26m1.482079716s) [noise=true , address=77.90.31.19:4841 , piece_id=AHAEDUPG3Y2WFR32QHH3D4IXVAIPUBPE55K2XUIAFPIE3OVH34LA, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.466436248 (195h26m1.482105858s) [noise=true , address=77.90.31.12:4589 , piece_id=KU2MPPPA4JDJY5LC2PHBTEW5ARMGCT7CIFMDLJ7KSLRHNK3U6QYA, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.466502890 (195h26m1.482172498s) [noise=true , address=145.79.227.42:4121 , piece_id=LNUVFNKRHVGAO4NAMWFUAGCZO2MXVYRSWFZ7CIIMLEQR3U6H5BUQ, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.467008781 (195h26m1.482678393s) [noise=true , address=145.79.227.12:4768 , piece_id=47YHQBI76KM4UYJBSX3PSMFCVF6VGWP5P42KZUTSORKXL72BMOEA, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.467190963 (195h26m1.482860573s) [noise=true , address=45.91.116.11:4118 , piece_id=2FVVXSBO2MQ7CET2XFGCZBKRTI7Z4LBAWVIVF4RREBLTZUUKKS3Q, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.467608100 (195h26m1.483277709s) [noise=true , address=145.79.227.3:4149 , piece_id=XQ4ZFYFWYL5QS2GW5Y37L3TTGK6MOKA7YNVB4LJPW55IQTXFLO7A, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.467635462 (195h26m1.483305073s) [noise=true , address=145.79.227.52:4450 , piece_id=4PXZJ5ILEJC4GDERWRB2JKFQLQPBHJFUBZ44PNVM5GJTYJRTEJEA, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.468066241 (195h26m1.483735851s) [noise=true , address=145.79.227.4:4191 , piece_id=RU4P2OOFHB5EUPHOZ2UMW37YMQACGAQU5HSCJW5BJJZFOJPCNSBA, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.468088435 (195h26m1.483758041s) [noise=true , address=145.79.227.45:4232 , piece_id=TSTJB23HJJRDDEDOBCLWBNBC6WPYFN47WDJE57KJDEL3YPMFG52Q, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.468520948 (195h26m1.484190555s) [noise=true , address=77.90.31.18:4805 , piece_id=HLT3A5QCZ7LVOK5N2EHRLN2HKFLK5CNINNB5HKPEDQNIJN7PEHUA, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.468627036 (195h26m1.484296646s) [noise=true , address=45.91.116.8:4046 , piece_id=JQSHJ4AJZDGF2HFF7OAI2R42CU4FGGA7O4AS2THUN3YQT2UBFVQQ, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.468959694 (195h26m1.484629301s) [noise=true , address=145.79.227.15:4875 , piece_id=SQ3LHFDNSRD7Q6IX5D3ODWWLBERCK65UAODP4P62HTQOI27VXHOA, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.469112974 (195h26m1.484782583s) [noise=true , address=145.79.227.39:4015 , piece_id=PMIDTKCZ2TR56KN4WCDSLNNFRFDQ4LZDVHNDAPUQD56ZS4UCUF4Q, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.469428536 (195h26m1.485098148s) [noise=true , address=145.79.227.7:4295 , piece_id=T2MAY5OQLNMVIFWCZX4R3PVRK2OBEWIZUUENT3Q5CBNN3DSO5GHA, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.469644653 (195h26m1.485314253s) [noise=true , address=145.79.227.41:4089 , piece_id=PIM7KNSZQGMDRA3CB2375T72HBODMOXW3PKMZ6AYJWVLEBDPUXLQ, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:58.470289518 (195h26m1.485959138s) [noise=true , address=45.91.116.14:4192 , piece_id=GOHSBBWYZ4YYO5PN3QRJN2Z2HNNCQN5VJEM7WS26O6QUPPNIBNMA, seg_pos={0 0}, upload=187733]: Uploading piece... Mar 18 19:36:59.087745205 (195h26m2.103414814s) [upload=187733]: Next returned nil segment Mar 18 19:36:59.087760721 (195h26m2.103430331s) [upload=187733]: Waiting for error group managing segments... Mar 18 19:36:59.221498315 (195h26m2.237167919s) [noise=true , address=45.91.116.14:4192 , piece_id=GOHSBBWYZ4YYO5PN3QRJN2Z2HNNCQN5VJEM7WS26O6QUPPNIBNMA, seg_pos={0 0}, upload=187733]: Done uploading piece. err: <nil> Mar 18 19:36:59.222264373 (195h26m2.237933988s) [noise=true , address=45.91.116.11:4118 , piece_id=2FVVXSBO2MQ7CET2XFGCZBKRTI7Z4LBAWVIVF4RREBLTZUUKKS3Q, seg_pos={0 0}, upload=187733]: Done uploading piece. err: <nil> Mar 18 19:36:59.222551225 (195h26m2.238220825s) [noise=true , address=77.90.31.12:4589 , piece_id=KU2MPPPA4JDJY5LC2PHBTEW5ARMGCT7CIFMDLJ7KSLRHNK3U6QYA, seg_pos={0 0}, upload=187733]: Done uploading piece. err: <nil> Mar 18 19:36:59.258816954 (195h26m2.274486558s) [noise=true , address=45.91.116.8:4046 , piece_id=JQSHJ4AJZDGF2HFF7OAI2R42CU4FGGA7O4AS2THUN3YQT2UBFVQQ, seg_pos={0 0}, upload=187733]: Done uploading piece. err: <nil> Mar 18 19:36:59.314775394 (195h26m2.330445002s) [noise=true , address=77.90.31.19:4841 , piece_id=AHAEDUPG3Y2WFR32QHH3D4IXVAIPUBPE55K2XUIAFPIE3OVH34LA, seg_pos={0 0}, upload=187733]: Done uploading piece. err: <nil> Mar 18 19:36:59.348692702 (195h26m2.364362302s) [noise=true , address=77.90.31.18:4805 , piece_id=HLT3A5QCZ7LVOK5N2EHRLN2HKFLK5CNINNB5HKPEDQNIJN7PEHUA, seg_pos={0 0}, upload=187733]: Done uploading piece. err: <nil> Mar 18 19:37:28.062993213 (195h26m31.078662826s) [noise=true , address=145.79.227.12:4768 , piece_id=47YHQBI76KM4UYJBSX3PSMFCVF6VGWP5P42KZUTSORKXL72BMOEA, seg_pos={0 0}, upload=187733]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: rpc: dial tcp 145.79.227.12:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:28.063032173 (195h26m31.078701782s) [noise=true , address=145.79.227.39:4015 , piece_id=PMIDTKCZ2TR56KN4WCDSLNNFRFDQ4LZDVHNDAPUQD56ZS4UCUF4Q, seg_pos={0 0}, upload=187733]: Done uploading piece. err: ecclient: upload failed (node:18etsqD4wpYunpwdbPETCAn3JGJawbFAK9mwTkVUYKuWB7ypQE, address:145.79.227.39:4015): protocol: rpc: dial tcp 145.79.227.39:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:28.063122456 (195h26m31.078792067s) [noise=true , address=145.79.227.41:4089 , piece_id=PIM7KNSZQGMDRA3CB2375T72HBODMOXW3PKMZ6AYJWVLEBDPUXLQ, seg_pos={0 0}, upload=187733]: Done uploading piece. err: ecclient: upload failed (node:12L2DTyJJ3v7KRjA424PVwQn9mpeq5Pq4ZTCerhrStgqgc3cVFk, address:145.79.227.41:4089): protocol: rpc: dial tcp 145.79.227.41:4089: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:28.063241482 (195h26m31.078911089s) [noise=true , address=145.79.227.52:4450 , piece_id=4PXZJ5ILEJC4GDERWRB2JKFQLQPBHJFUBZ44PNVM5GJTYJRTEJEA, seg_pos={0 0}, upload=187733]: Done uploading piece. err: ecclient: upload failed (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms, address:145.79.227.52:4450): protocol: rpc: dial tcp 145.79.227.52:4450: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:28.063305952 (195h26m31.078975557s) [noise=true , address=145.79.227.4:4191 , piece_id=RU4P2OOFHB5EUPHOZ2UMW37YMQACGAQU5HSCJW5BJJZFOJPCNSBA, seg_pos={0 0}, upload=187733]: Done uploading piece. err: ecclient: upload failed (node:1sVWXyKPnLTobAhk6GXZTTP67FbyY3XJ1e2svTiUEa3rrghd2q, address:145.79.227.4:4191): protocol: rpc: dial tcp 145.79.227.4:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:28.063323467 (195h26m31.078993082s) [noise=true , address=145.79.227.15:4875 , piece_id=SQ3LHFDNSRD7Q6IX5D3ODWWLBERCK65UAODP4P62HTQOI27VXHOA, seg_pos={0 0}, upload=187733]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Mar 18 19:37:28.063549391 (195h26m31.079219018s) [noise=true , address=145.79.227.45:4232 , piece_id=TSTJB23HJJRDDEDOBCLWBNBC6WPYFN47WDJE57KJDEL3YPMFG52Q, seg_pos={0 0}, upload=187733]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:37:28.063982376 (195h26m31.079652081s) [noise=true , address=145.79.227.42:4121 , piece_id=LNUVFNKRHVGAO4NAMWFUAGCZO2MXVYRSWFZ7CIIMLEQR3U6H5BUQ, seg_pos={0 0}, upload=187733]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:37:28.064242033 (195h26m31.079911643s) [noise=true , address=145.79.227.3:4149 , piece_id=XQ4ZFYFWYL5QS2GW5Y37L3TTGK6MOKA7YNVB4LJPW55IQTXFLO7A, seg_pos={0 0}, upload=187733]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:37:28.064273250 (195h26m31.079942859s) [noise=true , address=145.79.227.7:4295 , piece_id=T2MAY5OQLNMVIFWCZX4R3PVRK2OBEWIZUUENT3Q5CBNN3DSO5GHA, seg_pos={0 0}, upload=187733]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:37:28.064308747 (195h26m31.07997834s) [seg_pos={0 0}, upload=187733]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Mar 18 19:37:28.168596375 (195h26m31.184265978s) [upload=187733]: Done uploading segments... Mar 18 19:37:28.168662091 (195h26m31.184331704s) [upload=187733]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 30.109378768s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [.......................................................XXXXX] duration: 5.001220441s trace: https://jaeger.dev.storj.io/trace/352870b30043c2ca FAIL (2026-03-18 19:37:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.00125001s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XXX.] duration: 3.399301133s trace: https://jaeger.dev.storj.io/trace/56ad1b5361f648c4 FAIL (2026-03-18 19:37:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.399361499s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 1m0.001121847s trace: https://jaeger.dev.storj.io/trace/56ad1b5361f648c4 FAIL (2026-03-18 19:36:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001150862s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 30.114761281s trace: https://jaeger.dev.storj.io/trace/50aff7a361e86067 FAIL (2026-03-18 19:36:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 63RCJ5J37O5MDYDLHZDOOQ, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.242170469 (195h25m18.257840069s) [noise=true , address=45.91.116.10:4083 , piece_id=BB7Z3FPYBRDHLQ5CLBKK26FWNFCRV3ZSYOQNQAQTA5PAMFMGFVUQ, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.242192715 (195h25m18.257862317s) [noise=true , address=79.127.219.38:4821 , piece_id=JN2B2K5H4BNOMHCGKSETK6ME7N3WGJ7PA5SLECOYITPHEN5VKUKA, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.242692510 (195h25m18.258362114s) [noise=true , address=145.79.227.51:4412 , piece_id=TUS656KP7GDFFU54SBPCZVZ2NEVCGEAQPZ5N5T3IT6P4WVZ3XW2A, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.242726798 (195h25m18.258396401s) [noise=true , address=145.79.227.38:4982 , piece_id=35NS57PB2JATP2QQYH5LYAXPTMEUXSHCNVFP7GNVOLV2O23I4ZYA, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.243152909 (195h25m18.258822509s) [noise=true , address=145.79.227.13:4805 , piece_id=ZOYHDMJWE3VXKKW54QYJVIKNCYERX4EOZAZSOSMIJ276A3KACSRA, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.243178138 (195h25m18.258847743s) [noise=true , address=79.127.219.46:4893 , piece_id=JRMOVB7VUQOU2JCHHDEC32IVSUR6VJ4YRRJBTJFMPQPZKDWJATZQ, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.243685881 (195h25m18.259355493s) [noise=true , address=145.79.227.2:4118 , piece_id=2SNJTVQ4X45P6HZUNFULZSCTIVHBB5IPZQNOHF3GB5VD54PKD3QA, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.243761093 (195h25m18.259430701s) [noise=true , address=145.79.227.12:4771 , piece_id=EF6NITPQ5LGIU6WBQQGLOGIHHOWDKTAGOWOCIXM2O3NT7TJWXTEA, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.244189594 (195h25m18.259859204s) [noise=true , address=145.79.227.6:4263 , piece_id=N73HGMLPC5R5IPBAREXTV7NS76RBTPBBR5QKVZYLBJFPOA3RW7YA, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.244687112 (195h25m18.260356724s) [noise=true , address=145.79.227.39:4017 , piece_id=ZKFWDZKCYZJ4BN5DLLRVKZGQQVXEJP4ET5AAJGY2XDBYRRFB3UXA, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.245202519 (195h25m18.260872131s) [noise=true , address=145.79.227.21:4478 , piece_id=TTRRYWNO2IDLCNJZLPOD232PVSLVN2S4KCK4CRU3MRPFD3MY5IEQ, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.245708297 (195h25m18.2613779s) [noise=true , address=45.91.116.5:4301 , piece_id=VSVTMISGNHIWV75LR77CTKW3HEF57ZKXVSJWBFZTUU7KXPC2HBSA, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.246255991 (195h25m18.261925605s) [noise=true , address=145.79.227.18:4009 , piece_id=PLREAWHRETYSSRVI5ARZCTBCQEWP34N2VBKDVDFD4SUKIROT72EQ, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.247593686 (195h25m18.2632633s) [noise=true , address=145.79.227.4:4189 , piece_id=2WJSINTFLS5VXE4ZQNLAST5X5TJAIKW7WVDKJBEY7ZY6K4R5355A, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.249648256 (195h25m18.265317868s) [noise=true , address=145.79.227.30:4762 , piece_id=IVZVMWRIU4LIV2CPEEUQAJPIPYDJBSYVU447FNJPTPBV6VEDEEYQ, seg_pos={0 0}, upload=187720]: Uploading piece... Mar 18 19:36:15.854268253 (195h25m18.869937847s) [upload=187720]: Next returned nil segment Mar 18 19:36:15.854286297 (195h25m18.869955901s) [upload=187720]: Waiting for error group managing segments... Mar 18 19:36:15.996022602 (195h25m19.011692209s) [noise=true , address=79.127.219.38:4821 , piece_id=JN2B2K5H4BNOMHCGKSETK6ME7N3WGJ7PA5SLECOYITPHEN5VKUKA, seg_pos={0 0}, upload=187720]: Done uploading piece. err: <nil> Mar 18 19:36:15.998865861 (195h25m19.014535465s) [noise=true , address=45.91.116.10:4083 , piece_id=BB7Z3FPYBRDHLQ5CLBKK26FWNFCRV3ZSYOQNQAQTA5PAMFMGFVUQ, seg_pos={0 0}, upload=187720]: Done uploading piece. err: <nil> Mar 18 19:36:15.999137604 (195h25m19.014807238s) [noise=true , address=45.91.116.5:4301 , piece_id=VSVTMISGNHIWV75LR77CTKW3HEF57ZKXVSJWBFZTUU7KXPC2HBSA, seg_pos={0 0}, upload=187720]: Done uploading piece. err: <nil> Mar 18 19:36:16.041226508 (195h25m19.056896117s) [noise=true , address=79.127.219.46:4893 , piece_id=JRMOVB7VUQOU2JCHHDEC32IVSUR6VJ4YRRJBTJFMPQPZKDWJATZQ, seg_pos={0 0}, upload=187720]: Done uploading piece. err: <nil> Mar 18 19:36:44.832862113 (195h25m47.848531724s) [noise=true , address=145.79.227.8:4334 , piece_id=R6D4IV3XAST52WDU4YSDFOJOX4E5BL63RCJ5J37O5MDYDLHZDOOQ, seg_pos={0 0}, upload=187720]: Done uploading piece. err: ecclient: upload failed (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1, address:145.79.227.8:4334): protocol: rpc: dial tcp 145.79.227.8:4334: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:44.833127417 (195h25m47.848797023s) [noise=true , address=145.79.227.30:4762 , piece_id=IVZVMWRIU4LIV2CPEEUQAJPIPYDJBSYVU447FNJPTPBV6VEDEEYQ, seg_pos={0 0}, upload=187720]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: rpc: dial tcp 145.79.227.30:4762: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:44.833216330 (195h25m47.848886009s) [noise=true , address=145.79.227.12:4771 , piece_id=EF6NITPQ5LGIU6WBQQGLOGIHHOWDKTAGOWOCIXM2O3NT7TJWXTEA, seg_pos={0 0}, upload=187720]: Done uploading piece. err: ecclient: upload failed (node:1rcgrMxg2fRfvh7EA1eUehzD5rSH1krUd5W5eC8KTo7vjYWFLL, address:145.79.227.12:4771): protocol: rpc: dial tcp 145.79.227.12:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:44.833365521 (195h25m47.849035133s) [noise=true , address=145.79.227.51:4412 , piece_id=TUS656KP7GDFFU54SBPCZVZ2NEVCGEAQPZ5N5T3IT6P4WVZ3XW2A, seg_pos={0 0}, upload=187720]: Done uploading piece. err: ecclient: upload failed (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS, address:145.79.227.51:4412): protocol: rpc: dial tcp 145.79.227.51:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:44.833636225 (195h25m47.849305834s) [noise=true , address=145.79.227.38:4982 , piece_id=35NS57PB2JATP2QQYH5LYAXPTMEUXSHCNVFP7GNVOLV2O23I4ZYA, seg_pos={0 0}, upload=187720]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:145.79.227.38:4982): protocol: rpc: dial tcp 145.79.227.38:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:44.833661066 (195h25m47.849330679s) [noise=true , address=145.79.227.13:4805 , piece_id=ZOYHDMJWE3VXKKW54QYJVIKNCYERX4EOZAZSOSMIJ276A3KACSRA, seg_pos={0 0}, upload=187720]: Done uploading piece. err: ecclient: upload failed (node:12YEVsEwZwfWsnZFyaSkd9Dn53Tb9EFUyWEVDwTQeMJkcjwrj7x, address:145.79.227.13:4805): protocol: rpc: dial tcp 145.79.227.13:4805: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:44.834091438 (195h25m47.849761047s) [noise=true , address=145.79.227.21:4478 , piece_id=TTRRYWNO2IDLCNJZLPOD232PVSLVN2S4KCK4CRU3MRPFD3MY5IEQ, seg_pos={0 0}, upload=187720]: Done uploading piece. err: ecclient: upload failed (node:17kFjjJyvHApPUxUHJ6HeMEBCoo6xTbr8QizXjBEHwWj3mZUha, address:145.79.227.21:4478): protocol: rpc: dial tcp 145.79.227.21:4478: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:44.834296970 (195h25m47.849966579s) [noise=true , address=145.79.227.2:4118 , piece_id=2SNJTVQ4X45P6HZUNFULZSCTIVHBB5IPZQNOHF3GB5VD54PKD3QA, seg_pos={0 0}, upload=187720]: Done uploading piece. err: ecclient: upload failed (node:12ty5LHXwLmHC7nRPmVfj2wVEif6JQAQQ5Z5218vv1fKK5sD3zE, address:145.79.227.2:4118): protocol: rpc: dial tcp 145.79.227.2:4118: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:44.834336555 (195h25m47.850006165s) [noise=true , address=145.79.227.39:4017 , piece_id=ZKFWDZKCYZJ4BN5DLLRVKZGQQVXEJP4ET5AAJGY2XDBYRRFB3UXA, seg_pos={0 0}, upload=187720]: Done uploading piece. err: ecclient: upload failed (node:12ocB4w2b2owjyWVCcq59Ymo2prJfSFmK5YogQDjGBroF11Tegk, address:145.79.227.39:4017): protocol: rpc: dial tcp 145.79.227.39:4017: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:44.834528534 (195h25m47.850198143s) [noise=true , address=145.79.227.18:4009 , piece_id=PLREAWHRETYSSRVI5ARZCTBCQEWP34N2VBKDVDFD4SUKIROT72EQ, seg_pos={0 0}, upload=187720]: Done uploading piece. err: ecclient: upload failed (node:12n33Xie5Hk1vaKc279HvGnL2QBEYsNzieL2grpnANJpy7Qu2wb, address:145.79.227.18:4009): protocol: rpc: dial tcp 145.79.227.18:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:44.834699856 (195h25m47.850369469s) [noise=true , address=145.79.227.6:4263 , piece_id=N73HGMLPC5R5IPBAREXTV7NS76RBTPBBR5QKVZYLBJFPOA3RW7YA, seg_pos={0 0}, upload=187720]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:145.79.227.6:4263): protocol: rpc: dial tcp 145.79.227.6:4263: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:44.835196741 (195h25m47.850866356s) [noise=true , address=145.79.227.4:4189 , piece_id=2WJSINTFLS5VXE4ZQNLAST5X5TJAIKW7WVDKJBEY7ZY6K4R5355A, seg_pos={0 0}, upload=187720]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:145.79.227.4:4189): protocol: rpc: dial tcp 145.79.227.4:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:44.835258416 (195h25m47.850928019s) [seg_pos={0 0}, upload=187720]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Mar 18 19:36:44.946349252 (195h25m47.962018886s) [upload=187720]: Done uploading segments... Mar 18 19:36:44.946395386 (195h25m47.962064992s) [upload=187720]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 30.11477761s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [........................................................XXXX] duration: 5.000815589s trace: https://jaeger.dev.storj.io/trace/4dfd4316e495f8fa FAIL (2026-03-18 19:36:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000841019s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 30.114214655s trace: https://jaeger.dev.storj.io/trace/4b699c1079edff48 FAIL (2026-03-18 19:36:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.960388899 (195h24m47.976058507s) [noise=true , address=145.79.227.22:4512 , piece_id=BYQZDW5Z7HUR6KYTL5CGLQWVY2YETNDCB5F3ROXOJSBO5CXBXHGQ, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.960605787 (195h24m47.976275397s) [noise=true , address=145.79.227.33:4874 , piece_id=KIG6F6PBLF4O2X5DOB7FHFUO2QEB6NT6I77IX5LLLRADCUAZBZOQ, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.961042822 (195h24m47.976712429s) [noise=true , address=145.79.227.13:4803 , piece_id=HVROACX3UIDZ4Z7H4WL7MIPL766G6PYICYBPRD4WO3URBC7VT4YA, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.961438773 (195h24m47.977108378s) [noise=true , address=77.90.31.16:4736 , piece_id=L7GOQC7OT52RSXF65IZVZKEPPNT6RJ2RS2VTIJUJ3UKBDI2LEHKQ, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.962086134 (195h24m47.97775574s) [noise=true , address=145.79.227.3:4149 , piece_id=I62P4XFJH2OZ3DWNKDAG5YFIAXZUFVJ47YXQI7ZYTHS5VBU3DPRQ, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.962377472 (195h24m47.978047079s) [noise=true , address=145.79.227.8:4332 , piece_id=E7RHYHTIG5NBP4WABGTB3SVTCT3ZLK6JXFBMGZ272OUIHZL5ECSQ, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.962467793 (195h24m47.978137394s) [noise=true , address=145.79.227.10:4404 , piece_id=HM7ERL5NUQWXEBLSQMFGLUY5ZZXGE2NIK46USD6PKOYDGWVK6VIQ, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.962834491 (195h24m47.978504091s) [noise=true , address=145.79.227.11:4733 , piece_id=YTUL25DNUAOYU2BURFOLKCPAGINZEPG4XHMYJ63BJMXO6ZMW6JSA, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.962907954 (195h24m47.978577564s) [noise=true , address=145.79.227.23:4550 , piece_id=GWABIYW43ZC5LFKNBLOUDRUZPVZMELQ4MVWX4SDXS74FPQIQUPDA, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.963267576 (195h24m47.978937184s) [noise=true , address=145.79.227.4:4190 , piece_id=T2DDNNUJ4VJBMN2AARPAI7YR52NXVAX42L6ICV6MOQI2VFTXIJ4A, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.963393393 (195h24m47.979062991s) [noise=true , address=45.91.116.8:4047 , piece_id=I7DALF4JFVYUXZSIFNWLS3XHRRGBISY2H7CIVCQLAOZ37QRTGRVA, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.963660171 (195h24m47.979329774s) [noise=true , address=45.91.116.11:4152 , piece_id=OAQPVUDPLDDPYJZXPYLWUMCM7XG7YJBKH5YJWAFHYPVOYB3DCXLA, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.964006687 (195h24m47.979676291s) [noise=true , address=145.79.227.31:4802 , piece_id=DIGQZ3ZNB3QOFDMFPNZI6OGYISCPIBUBJPKXNNOZ3DM2UYIUAEZQ, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:44.964953899 (195h24m47.980623508s) [noise=true , address=145.79.227.50:4376 , piece_id=THWPEXJ6YZJ3TJ7RAVEFMBGDJ6QQIYP2ZFSCSZWGLH67IF52JJSA, seg_pos={0 0}, upload=187713]: Uploading piece... Mar 18 19:35:45.559147028 (195h24m48.574816634s) [upload=187713]: Next returned nil segment Mar 18 19:35:45.559165609 (195h24m48.574835222s) [upload=187713]: Waiting for error group managing segments... Mar 18 19:35:45.693006753 (195h24m48.708676357s) [noise=true , address=77.90.31.16:4736 , piece_id=L7GOQC7OT52RSXF65IZVZKEPPNT6RJ2RS2VTIJUJ3UKBDI2LEHKQ, seg_pos={0 0}, upload=187713]: Done uploading piece. err: <nil> Mar 18 19:35:45.726451020 (195h24m48.742120622s) [noise=true , address=45.91.116.11:4152 , piece_id=OAQPVUDPLDDPYJZXPYLWUMCM7XG7YJBKH5YJWAFHYPVOYB3DCXLA, seg_pos={0 0}, upload=187713]: Done uploading piece. err: <nil> Mar 18 19:35:45.746614895 (195h24m48.762284495s) [noise=true , address=45.91.116.8:4047 , piece_id=I7DALF4JFVYUXZSIFNWLS3XHRRGBISY2H7CIVCQLAOZ37QRTGRVA, seg_pos={0 0}, upload=187713]: Done uploading piece. err: <nil> Mar 18 19:36:14.570588666 (195h25m17.586258295s) [noise=true , address=145.79.227.23:4550 , piece_id=GWABIYW43ZC5LFKNBLOUDRUZPVZMELQ4MVWX4SDXS74FPQIQUPDA, seg_pos={0 0}, upload=187713]: Done uploading piece. err: ecclient: upload failed (node:12kn3uD4MCE56bN4MYYsGjDJd2qebgZ6VB6mavL711M8hoU8fbE, address:145.79.227.23:4550): protocol: rpc: dial tcp 145.79.227.23:4550: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:14.570652444 (195h25m17.586322055s) [noise=true , address=145.79.227.26:4657 , piece_id=LOWQSWM5SFG546QBH7B5SC5B6YXDUPTMZTQKWKMEADJSUQ4ZCI4Q, seg_pos={0 0}, upload=187713]: Done uploading piece. err: ecclient: upload failed (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq, address:145.79.227.26:4657): protocol: rpc: dial tcp 145.79.227.26:4657: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:14.570940586 (195h25m17.586610195s) [noise=true , address=145.79.227.2:4115 , piece_id=PC3OQ3P2FJFKW2O4AX5ATOUG3C6OGYRSSW3BBM4BXDVEDMNYADWQ, seg_pos={0 0}, upload=187713]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:14.571306770 (195h25m17.586976379s) [noise=true , address=145.79.227.50:4376 , piece_id=THWPEXJ6YZJ3TJ7RAVEFMBGDJ6QQIYP2ZFSCSZWGLH67IF52JJSA, seg_pos={0 0}, upload=187713]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: rpc: dial tcp 145.79.227.50:4376: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:14.571386930 (195h25m17.587056539s) [noise=true , address=145.79.227.11:4733 , piece_id=YTUL25DNUAOYU2BURFOLKCPAGINZEPG4XHMYJ63BJMXO6ZMW6JSA, seg_pos={0 0}, upload=187713]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:14.571603555 (195h25m17.587273167s) [noise=true , address=145.79.227.10:4404 , piece_id=HM7ERL5NUQWXEBLSQMFGLUY5ZZXGE2NIK46USD6PKOYDGWVK6VIQ, seg_pos={0 0}, upload=187713]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: rpc: dial tcp 145.79.227.10:4404: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:14.571671175 (195h25m17.587340785s) [noise=true , address=145.79.227.22:4512 , piece_id=BYQZDW5Z7HUR6KYTL5CGLQWVY2YETNDCB5F3ROXOJSBO5CXBXHGQ, seg_pos={0 0}, upload=187713]: Done uploading piece. err: ecclient: upload failed (node:12cnVqYYfo8b6RvRtMykzh8mU7UAqxcqYvWe7H8G7k8DqtNLAR4, address:145.79.227.22:4512): protocol: rpc: dial tcp 145.79.227.22:4512: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:14.571890428 (195h25m17.587560106s) [noise=true , address=145.79.227.33:4874 , piece_id=KIG6F6PBLF4O2X5DOB7FHFUO2QEB6NT6I77IX5LLLRADCUAZBZOQ, seg_pos={0 0}, upload=187713]: Done uploading piece. err: ecclient: upload failed (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq, address:145.79.227.33:4874): protocol: rpc: dial tcp 145.79.227.33:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:14.571944138 (195h25m17.587613741s) [noise=true , address=145.79.227.13:4803 , piece_id=HVROACX3UIDZ4Z7H4WL7MIPL766G6PYICYBPRD4WO3URBC7VT4YA, seg_pos={0 0}, upload=187713]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: rpc: dial tcp 145.79.227.13:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:14.572050204 (195h25m17.587719813s) [noise=true , address=145.79.227.3:4149 , piece_id=I62P4XFJH2OZ3DWNKDAG5YFIAXZUFVJ47YXQI7ZYTHS5VBU3DPRQ, seg_pos={0 0}, upload=187713]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:14.572064487 (195h25m17.587734097s) [noise=true , address=145.79.227.8:4332 , piece_id=E7RHYHTIG5NBP4WABGTB3SVTCT3ZLK6JXFBMGZ272OUIHZL5ECSQ, seg_pos={0 0}, upload=187713]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:145.79.227.8:4332): protocol: rpc: dial tcp 145.79.227.8:4332: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:14.572283945 (195h25m17.587953553s) [noise=true , address=145.79.227.4:4190 , piece_id=T2DDNNUJ4VJBMN2AARPAI7YR52NXVAX42L6ICV6MOQI2VFTXIJ4A, seg_pos={0 0}, upload=187713]: Done uploading piece. err: ecclient: upload failed (node:1eNJzs5Dc7Nv7imoQXZDp2mEVWjD8o4viGVkFvuAeZ1arynxu4, address:145.79.227.4:4190): protocol: rpc: dial tcp 145.79.227.4:4190: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Mar 18 19:36:14.572455555 (195h25m17.588125174s) [noise=true , address=145.79.227.31:4802 , piece_id=DIGQZ3ZNB3QOFDMFPNZI6OGYISCPIBUBJPKXNNOZ3DM2UYIUAEZQ, seg_pos={0 0}, upload=187713]: Done uploading piece. err: ecclient: upload failed (node:1iMEuZv8uHpYjTqP9JVpznKPPdEyyy4xMTsgVUGMuCjZkV4cri, address:145.79.227.31:4802): protocol: rpc: dial tcp 145.79.227.31:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:14.572507218 (195h25m17.588176805s) [seg_pos={0 0}, upload=187713]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Mar 18 19:36:14.682936024 (195h25m17.698605639s) [upload=187713]: Done uploading segments... Mar 18 19:36:14.682992025 (195h25m17.698661637s) [upload=187713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 30.114241479s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.........................................................XXX] duration: 5.001350676s trace: https://jaeger.dev.storj.io/trace/7b575236bb53a547 FAIL (2026-03-18 19:36:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001375049s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 3.206098435s trace: https://jaeger.dev.storj.io/trace/1dbbc2c03b82519a FAIL (2026-03-18 19:36:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 682742s) [noise=true , address=45.91.116.14:4198 , piece_id=P4ATZCI44XZCYV4ZHHEWRIBZTI25N2YPLUQACKQSFHU5ZIDNAM4A, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.243123447 (195h23m20.258793057s) [noise=true , address=145.79.227.25:4645 , piece_id=FYSRVAFVZL424CYO6DSGFGWJLNZW6RWHNWVX6HSPTDZSYVLPBYOA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.244312753 (195h23m20.259982353s) [noise=true , address=145.79.227.23:4580 , piece_id=TWT5DAXYGGEXAFP7EOXDBUJ3JFGDXCWAG46O2NMAAN2DZVRRNI4A, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.244446784 (195h23m20.260116392s) [noise=true , address=45.91.116.13:4179 , piece_id=RMEDLCAM3SED327FL2ENLJFAYUX6WB3BMU364UB3RHE2YY7NJ42A, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.244846632 (195h23m20.260516238s) [noise=true , address=145.79.227.22:4519 , piece_id=N5H4SKH46OFD47RN3TVB2OSWVCPY7KRMXHYXP4FO27JW54IQN7RQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.244928519 (195h23m20.260598129s) [noise=true , address=193.149.164.67:4029 , piece_id=AORLLFAMQKQ6WH3IIXYCS5R5EJW2CC7X4K5SWCXNXB27T6JFZWZA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.245353921 (195h23m20.26102353s) [noise=true , address=167.17.72.71:4816 , piece_id=N2SOF6MSXA2NYKTPSVDSMDSNKLZYYAMTK3TARDOB6QM3S24MMROA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.245385841 (195h23m20.261055658s) [noise=true , address=145.79.227.49:4358 , piece_id=NGBDKRHG4KJJQJIWWSLPZTFVDYWHAQJ73W55ZW5MPBRIC4LDZTOQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.246153076 (195h23m20.261822681s) [noise=true , address=77.90.31.11:4575 , piece_id=P6B3YXSQQ4YRORDLS5POQY4ZCWPNTC554UEZD7PAKHFWJ2QGTENQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.246634250 (195h23m20.262303852s) [noise=true , address=145.79.227.24:4603 , piece_id=GXKKSMCUK4ZI3IFHB5XVEVVGGDBW3VDZNS3H72BZJR3NOL2ABIVQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.246676628 (195h23m20.262346237s) [noise=true , address=167.17.72.69:4740 , piece_id=6Q5JOGHJV5NETPZ53INLNDL64KUTOH6FKQNX23UOQ2TBBPSSDYJQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.247101368 (195h23m20.26277097s) [noise=true , address=193.149.164.68:4061 , piece_id=I2YF5IQTOS3KSWL3GBSSBOCR4J4V7BT4PN3VWZGUSGMBAPILSHPQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.247375193 (195h23m20.263044797s) [noise=true , address=79.127.219.38:4851 , piece_id=F4EGHAVII6L4EY626YFAAPHTUD22HMAJCZZXJQDCQ5D7GTICUOTA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.247656193 (195h23m20.263325783s) [noise=true , address=145.79.227.33:4883 , piece_id=4KT3BZ5RLJU6VZ3JSPYJ5GV3TQPCMRMQDQQCLFMSSTTHXPQZ2Z6Q, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.247789006 (195h23m20.263458598s) [noise=true , address=79.127.205.237:4237 , piece_id=2TFJA4FUPBQP56URZ3DVRC6BWDIGJOEPOBI6TXVSVEUW2SPVII2Q, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.248224855 (195h23m20.263894465s) [noise=true , address=145.79.227.28:4754 , piece_id=4OYL73BSJ7VQUHI6KXS47AYGX34SHXZPD6HWCCLUXZWOC7U7EBZQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.248582486 (195h23m20.264252076s) [noise=true , address=145.79.227.43:4176 , piece_id=NLWMKDHQ4P5GBCD6OWAL5DCN6U27BBARHMEY2TFEQ55CD7UN3LMA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.248836068 (195h23m20.264505661s) [noise=true , address=77.90.31.14:4680 , piece_id=ZQM2LQ6DIW4I2KBPANJG6UXDEX54EQWCT74HYPUFFZJ3NYV4FWBA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.800000668 (195h23m20.815670276s) [upload=187687]: Next returned nil segment Mar 18 19:34:17.800018306 (195h23m20.815687917s) [upload=187687]: Waiting for error group managing segments... Mar 18 19:34:17.946303744 (195h23m20.961973334s) [noise=true , address=45.91.116.13:4179 , piece_id=RMEDLCAM3SED327FL2ENLJFAYUX6WB3BMU364UB3RHE2YY7NJ42A, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:17.958150371 (195h23m20.973819975s) [noise=true , address=77.90.31.11:4575 , piece_id=P6B3YXSQQ4YRORDLS5POQY4ZCWPNTC554UEZD7PAKHFWJ2QGTENQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:17.974046250 (195h23m20.989715855s) [noise=true , address=79.127.219.38:4851 , piece_id=F4EGHAVII6L4EY626YFAAPHTUD22HMAJCZZXJQDCQ5D7GTICUOTA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:17.978120808 (195h23m20.993790412s) [noise=true , address=45.91.116.14:4198 , piece_id=P4ATZCI44XZCYV4ZHHEWRIBZTI25N2YPLUQACKQSFHU5ZIDNAM4A, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:17.981186396 (195h23m20.996855996s) [noise=true , address=77.90.31.14:4680 , piece_id=ZQM2LQ6DIW4I2KBPANJG6UXDEX54EQWCT74HYPUFFZJ3NYV4FWBA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.301445669 (195h23m21.317115272s) [noise=true , address=167.17.72.71:4816 , piece_id=N2SOF6MSXA2NYKTPSVDSMDSNKLZYYAMTK3TARDOB6QM3S24MMROA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.331480464 (195h23m21.347150077s) [noise=true , address=167.17.72.69:4740 , piece_id=6Q5JOGHJV5NETPZ53INLNDL64KUTOH6FKQNX23UOQ2TBBPSSDYJQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.480437207 (195h23m21.49610682s) [noise=true , address=193.149.164.68:4061 , piece_id=I2YF5IQTOS3KSWL3GBSSBOCR4J4V7BT4PN3VWZGUSGMBAPILSHPQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.506341021 (195h23m21.522010628s) [noise=true , address=193.149.164.67:4029 , piece_id=AORLLFAMQKQ6WH3IIXYCS5R5EJW2CC7X4K5SWCXNXB27T6JFZWZA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.682660820 (195h23m21.698330443s) [noise=true , address=79.127.205.237:4237 , piece_id=2TFJA4FUPBQP56URZ3DVRC6BWDIGJOEPOBI6TXVSVEUW2SPVII2Q, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:35:16.852210743 (195h24m19.867880349s) [noise=true , address=145.79.227.33:4883 , piece_id=4KT3BZ5RLJU6VZ3JSPYJ5GV3TQPCMRMQDQQCLFMSSTTHXPQZ2Z6Q, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:12cx1avL4umDm2pzJS1rNZqeDZjZMWnMtNsyNTdMdkxhpa1ByUN, address:145.79.227.33:4883): protocol: rpc: dial tcp 145.79.227.33:4883: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.852525149 (195h24m19.868194752s) [noise=true , address=145.79.227.43:4176 , piece_id=NLWMKDHQ4P5GBCD6OWAL5DCN6U27BBARHMEY2TFEQ55CD7UN3LMA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:1Yi5AibNiCNV8vUb9MKgooHCw8Nr79m9HqTrkrYybNZmatg9PD, address:145.79.227.43:4176): protocol: rpc: dial tcp 145.79.227.43:4176: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.852540920 (195h24m19.868210529s) [noise=true , address=145.79.227.23:4580 , piece_id=TWT5DAXYGGEXAFP7EOXDBUJ3JFGDXCWAG46O2NMAAN2DZVRRNI4A, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:12569SWj8WmR8tWoBXk7XJK1KS6BZMcYHrQjPKGfhj21xFPQ1W3, address:145.79.227.23:4580): protocol: rpc: dial tcp 145.79.227.23:4580: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.852795830 (195h24m19.868465442s) [noise=true , address=145.79.227.22:4519 , piece_id=N5H4SKH46OFD47RN3TVB2OSWVCPY7KRMXHYXP4FO27JW54IQN7RQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:1p5DYJfayCq68BQGnSEAoZCri8TWpST5WNcx5QFvzcxLyFamnj, address:145.79.227.22:4519): protocol: rpc: dial tcp 145.79.227.22:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.852913053 (195h24m19.868582662s) [noise=true , address=145.79.227.24:4603 , piece_id=GXKKSMCUK4ZI3IFHB5XVEVVGGDBW3VDZNS3H72BZJR3NOL2ABIVQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:145.79.227.24:4603): protocol: rpc: dial tcp 145.79.227.24:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.853081685 (195h24m19.868751288s) [noise=true , address=145.79.227.49:4358 , piece_id=NGBDKRHG4KJJQJIWWSLPZTFVDYWHAQJ73W55ZW5MPBRIC4LDZTOQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:1dwCNEcA9o51ZN3UaN3Frxdawwf8DXBj86iQMywjW8XP4ESmw1, address:145.79.227.49:4358): protocol: rpc: dial tcp 145.79.227.49:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.853184716 (195h24m19.868854329s) [noise=true , address=145.79.227.25:4645 , piece_id=FYSRVAFVZL424CYO6DSGFGWJLNZW6RWHNWVX6HSPTDZSYVLPBYOA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:12NPZfT1KXFLcfPTjX8zLQ4gpagKYosJbvCTKu9sK7989hDjTPm, address:145.79.227.25:4645): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:35:16.853380516 (195h24m19.869050119s) [noise=true , address=145.79.227.28:4754 , piece_id=4OYL73BSJ7VQUHI6KXS47AYGX34SHXZPD6HWCCLUXZWOC7U7EBZQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:1HB1v3ZURPMSYsbHBjJs1neE7HQtLS1KmeLwbKK2sTKAZcXPhi, address:145.79.227.28:4754): protocol: rpc: dial tcp 145.79.227.28:4754: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.853414478 (195h24m19.86908408s) [seg_pos={0 0}, upload=187687]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Mar 18 19:35:16.961105067 (195h24m19.976774674s) [upload=187687]: Done uploading segments... Mar 18 19:35:16.961154576 (195h24m19.976824177s) [upload=187687]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 3.206123888s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.001477293s trace: https://jaeger.dev.storj.io/trace/6cf1e3e1915a0b94 FAIL (2026-03-18 19:35:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001503121s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.118719735s trace: https://jaeger.dev.storj.io/trace/47b964f260b3588d FAIL (2026-03-18 19:35:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== WG6ZLE4ZZ63RXM4RA, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:10.707671944 (195h24m13.723341552s) [noise=true , address=145.79.227.32:4833 , piece_id=L7YYC7DTMMF6LMIGRZ2CJJYOFBDBA6XJGPGM5XEO2TGSJ5YITQYA, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:10.710067538 (195h24m13.725737154s) [noise=true , address=145.79.227.21:4477 , piece_id=SOI3HC7IF55JH5EJHAMW4KLVNN6VC65AHKNYYUV5WQ7WRVMSXPOA, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:10.729106614 (195h24m13.744776223s) [noise=true , address=145.79.227.51:4414 , piece_id=ZER7VRRFJR7GVARMRJPF5NBNORATZRC6YJ4JVCGTHONKGI3ULEWA, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:10.732238312 (195h24m13.747907917s) [noise=true , address=145.79.227.3:4153 , piece_id=QAR5MNBIX6IOLKS4PIFCEXCF37S6UNUVR6IGS7IMNUOOTRJYDMAA, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:10.736890302 (195h24m13.752559907s) [noise=true , address=145.79.227.40:4050 , piece_id=KQVF3RC5FO3VIKKYKGPRGIEMHGOJP3WEXJ23RRYQDHFOZW2PXEZQ, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:10.739431608 (195h24m13.75510121s) [noise=true , address=145.79.227.19:4083 , piece_id=QUWVUMFRWFPOBDIUN74A3FGZABUHMBM36LQT6FTRE2UQSJNGOG7A, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:10.742395151 (195h24m13.758064752s) [noise=true , address=145.79.227.10:4405 , piece_id=CZRZADZXPKPF44QRNAYNNJUTG4PI2SB43J4JHBFHYFDB6YA64GKA, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:10.745139013 (195h24m13.760808624s) [noise=true , address=145.79.227.39:4016 , piece_id=NG4JZDDJWBNP5EZWFK4NBO3PXJJPRYFVPDK3UKHJBCEPDD6QCI7Q, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:10.747676427 (195h24m13.763346038s) [noise=true , address=145.79.227.48:4304 , piece_id=KVQXZOVEX3PPBS24OEQEOU2BF5MLE3Z563GNXAM4SIA6SDKAEWKQ, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:10.751125517 (195h24m13.766795126s) [noise=true , address=145.79.227.2:4118 , piece_id=PDI7IAGZXDS6R2NMZHZL7XOBT3KYFOO6CNPRY6F7EHAPGKPDHB3Q, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:10.754372959 (195h24m13.770042561s) [noise=true , address=145.79.227.46:4604 , piece_id=4APWCZJGQBJSELS4FC5OU354CTEA464AV6BVVXXSLVY2ZACIAZ2A, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:10.757561206 (195h24m13.773230818s) [noise=true , address=145.79.227.14:4839 , piece_id=JE6YJWBUFLDFG65WRGLTBFJOHRSJ6U75H4EPSIXUISS2QI6FU4VA, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:10.761495839 (195h24m13.77716544s) [noise=true , address=45.91.116.9:4261 , piece_id=L5S53DTADEJV3OVD6GNSZAJTSFMVBPBOE3A7VMVJJAFPOLKDTA2A, seg_pos={0 0}, upload=187704]: Uploading piece... Mar 18 19:35:11.375927041 (195h24m14.391596632s) [upload=187704]: Next returned nil segment Mar 18 19:35:11.375944008 (195h24m14.391613613s) [upload=187704]: Waiting for error group managing segments... Mar 18 19:35:11.531708172 (195h24m14.547377772s) [noise=true , address=45.91.116.9:4261 , piece_id=L5S53DTADEJV3OVD6GNSZAJTSFMVBPBOE3A7VMVJJAFPOLKDTA2A, seg_pos={0 0}, upload=187704]: Done uploading piece. err: <nil> Mar 18 19:35:11.534678239 (195h24m14.550347839s) [noise=true , address=77.90.31.10:4521 , piece_id=UJODTKKBACPPLEGGTABR7XDA64LB2V5EBAFM2BPROX5DMDQ5A2KA, seg_pos={0 0}, upload=187704]: Done uploading piece. err: <nil> Mar 18 19:35:40.223965224 (195h24m43.239634833s) [noise=true , address=145.79.227.14:4839 , piece_id=JE6YJWBUFLDFG65WRGLTBFJOHRSJ6U75H4EPSIXUISS2QI6FU4VA, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.224298672 (195h24m43.239968281s) [noise=true , address=145.79.227.10:4405 , piece_id=CZRZADZXPKPF44QRNAYNNJUTG4PI2SB43J4JHBFHYFDB6YA64GKA, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:1MCHYW29ZQjE2fHN8P98hDfNRLsjiVqabF86TXz5nZRBPhbGy4, address:145.79.227.10:4405): protocol: rpc: dial tcp 145.79.227.10:4405: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.224613230 (195h24m43.24028284s) [noise=true , address=145.79.227.39:4016 , piece_id=NG4JZDDJWBNP5EZWFK4NBO3PXJJPRYFVPDK3UKHJBCEPDD6QCI7Q, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:1jC98jEWn6K8jFHPA9knJeHAkuNEmcS7ThxUvbySDfbq6FmKVR, address:145.79.227.39:4016): protocol: rpc: dial tcp 145.79.227.39:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.224922032 (195h24m43.240591644s) [noise=true , address=145.79.227.2:4118 , piece_id=PDI7IAGZXDS6R2NMZHZL7XOBT3KYFOO6CNPRY6F7EHAPGKPDHB3Q, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:12ty5LHXwLmHC7nRPmVfj2wVEif6JQAQQ5Z5218vv1fKK5sD3zE, address:145.79.227.2:4118): protocol: rpc: dial tcp 145.79.227.2:4118: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.225230550 (195h24m43.240900162s) [noise=true , address=145.79.227.37:4568 , piece_id=IZTDU6SPM4SLE4QASTMEWIADOQI42HNCS6WWG6ZLE4ZZ63RXM4RA, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:1zJigHghNntzHNWim5RnWcMPfeUAbqsrvrfWretYcyEkXbSYzJ, address:145.79.227.37:4568): protocol: rpc: dial tcp 145.79.227.37:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.225534263 (195h24m43.241203873s) [noise=true , address=145.79.227.32:4833 , piece_id=L7YYC7DTMMF6LMIGRZ2CJJYOFBDBA6XJGPGM5XEO2TGSJ5YITQYA, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: rpc: dial tcp 145.79.227.32:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.226095253 (195h24m43.241764864s) [noise=true , address=145.79.227.21:4477 , piece_id=SOI3HC7IF55JH5EJHAMW4KLVNN6VC65AHKNYYUV5WQ7WRVMSXPOA, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY, address:145.79.227.21:4477): protocol: rpc: dial tcp 145.79.227.21:4477: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.226419305 (195h24m43.242088918s) [noise=true , address=145.79.227.51:4414 , piece_id=ZER7VRRFJR7GVARMRJPF5NBNORATZRC6YJ4JVCGTHONKGI3ULEWA, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:1Gh17XmEkfiFPqqR1ASnC7LoMz6ik3a5EHitxG67GVQkDxAHc2, address:145.79.227.51:4414): protocol: rpc: dial tcp 145.79.227.51:4414: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.226707083 (195h24m43.242376696s) [noise=true , address=145.79.227.19:4083 , piece_id=QUWVUMFRWFPOBDIUN74A3FGZABUHMBM36LQT6FTRE2UQSJNGOG7A, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:12KZ5a6CQ9tUTrC6EYE7M8yg6p9fK95CN8zQLQmP4tdyz4rBfGh, address:145.79.227.19:4083): protocol: rpc: dial tcp 145.79.227.19:4083: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.227015661 (195h24m43.242685273s) [noise=true , address=145.79.227.48:4304 , piece_id=KVQXZOVEX3PPBS24OEQEOU2BF5MLE3Z563GNXAM4SIA6SDKAEWKQ, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.227518322 (195h24m43.243187933s) [noise=true , address=145.79.227.3:4153 , piece_id=QAR5MNBIX6IOLKS4PIFCEXCF37S6UNUVR6IGS7IMNUOOTRJYDMAA, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:124myE1wFhvjA5paJy5tr2hRe8jZCoEdbJLdBwvss1ZnT4GvtZv, address:145.79.227.3:4153): protocol: rpc: dial tcp 145.79.227.3:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.227781108 (195h24m43.24345072s) [noise=true , address=145.79.227.40:4050 , piece_id=KQVF3RC5FO3VIKKYKGPRGIEMHGOJP3WEXJ23RRYQDHFOZW2PXEZQ, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:12Vw9PtVFLQPpm2uGV55tCoK7VfopaBU4EcWNBUyqhNeJjgjG64, address:145.79.227.40:4050): protocol: rpc: dial tcp 145.79.227.40:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.228130519 (195h24m43.243800132s) [noise=true , address=145.79.227.46:4604 , piece_id=4APWCZJGQBJSELS4FC5OU354CTEA464AV6BVVXXSLVY2ZACIAZ2A, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.230066039 (195h24m43.245735653s) [noise=true , address=145.79.227.25:4621 , piece_id=LNEFRNE32YNG2A72LXAM7HFL52NKK6K7BQOZE4U62CQSMT7TMKXA, seg_pos={0 0}, upload=187704]: Done uploading piece. err: ecclient: upload failed (node:12VY2FEQWX4WPXwgMVBKoANFgqFd1npRBUG8pfQKWVB6gJw96AS, address:145.79.227.25:4621): protocol: rpc: dial tcp 145.79.227.25:4621: i/o timeout; context deadline exceeded Mar 18 19:35:40.230114391 (195h24m43.245783982s) [seg_pos={0 0}, upload=187704]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Mar 18 19:35:40.340780667 (195h24m43.356450279s) [upload=187704]: Done uploading segments... Mar 18 19:35:40.340849417 (195h24m43.356519025s) [upload=187704]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 30.118744446s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 3.265491178s trace: https://jaeger.dev.storj.io/trace/1dbbc2c03b82519a FAIL (2026-03-18 19:35:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 682742s) [noise=true , address=45.91.116.14:4198 , piece_id=P4ATZCI44XZCYV4ZHHEWRIBZTI25N2YPLUQACKQSFHU5ZIDNAM4A, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.243123447 (195h23m20.258793057s) [noise=true , address=145.79.227.25:4645 , piece_id=FYSRVAFVZL424CYO6DSGFGWJLNZW6RWHNWVX6HSPTDZSYVLPBYOA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.244312753 (195h23m20.259982353s) [noise=true , address=145.79.227.23:4580 , piece_id=TWT5DAXYGGEXAFP7EOXDBUJ3JFGDXCWAG46O2NMAAN2DZVRRNI4A, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.244446784 (195h23m20.260116392s) [noise=true , address=45.91.116.13:4179 , piece_id=RMEDLCAM3SED327FL2ENLJFAYUX6WB3BMU364UB3RHE2YY7NJ42A, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.244846632 (195h23m20.260516238s) [noise=true , address=145.79.227.22:4519 , piece_id=N5H4SKH46OFD47RN3TVB2OSWVCPY7KRMXHYXP4FO27JW54IQN7RQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.244928519 (195h23m20.260598129s) [noise=true , address=193.149.164.67:4029 , piece_id=AORLLFAMQKQ6WH3IIXYCS5R5EJW2CC7X4K5SWCXNXB27T6JFZWZA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.245353921 (195h23m20.26102353s) [noise=true , address=167.17.72.71:4816 , piece_id=N2SOF6MSXA2NYKTPSVDSMDSNKLZYYAMTK3TARDOB6QM3S24MMROA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.245385841 (195h23m20.261055658s) [noise=true , address=145.79.227.49:4358 , piece_id=NGBDKRHG4KJJQJIWWSLPZTFVDYWHAQJ73W55ZW5MPBRIC4LDZTOQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.246153076 (195h23m20.261822681s) [noise=true , address=77.90.31.11:4575 , piece_id=P6B3YXSQQ4YRORDLS5POQY4ZCWPNTC554UEZD7PAKHFWJ2QGTENQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.246634250 (195h23m20.262303852s) [noise=true , address=145.79.227.24:4603 , piece_id=GXKKSMCUK4ZI3IFHB5XVEVVGGDBW3VDZNS3H72BZJR3NOL2ABIVQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.246676628 (195h23m20.262346237s) [noise=true , address=167.17.72.69:4740 , piece_id=6Q5JOGHJV5NETPZ53INLNDL64KUTOH6FKQNX23UOQ2TBBPSSDYJQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.247101368 (195h23m20.26277097s) [noise=true , address=193.149.164.68:4061 , piece_id=I2YF5IQTOS3KSWL3GBSSBOCR4J4V7BT4PN3VWZGUSGMBAPILSHPQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.247375193 (195h23m20.263044797s) [noise=true , address=79.127.219.38:4851 , piece_id=F4EGHAVII6L4EY626YFAAPHTUD22HMAJCZZXJQDCQ5D7GTICUOTA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.247656193 (195h23m20.263325783s) [noise=true , address=145.79.227.33:4883 , piece_id=4KT3BZ5RLJU6VZ3JSPYJ5GV3TQPCMRMQDQQCLFMSSTTHXPQZ2Z6Q, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.247789006 (195h23m20.263458598s) [noise=true , address=79.127.205.237:4237 , piece_id=2TFJA4FUPBQP56URZ3DVRC6BWDIGJOEPOBI6TXVSVEUW2SPVII2Q, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.248224855 (195h23m20.263894465s) [noise=true , address=145.79.227.28:4754 , piece_id=4OYL73BSJ7VQUHI6KXS47AYGX34SHXZPD6HWCCLUXZWOC7U7EBZQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.248582486 (195h23m20.264252076s) [noise=true , address=145.79.227.43:4176 , piece_id=NLWMKDHQ4P5GBCD6OWAL5DCN6U27BBARHMEY2TFEQ55CD7UN3LMA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.248836068 (195h23m20.264505661s) [noise=true , address=77.90.31.14:4680 , piece_id=ZQM2LQ6DIW4I2KBPANJG6UXDEX54EQWCT74HYPUFFZJ3NYV4FWBA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.800000668 (195h23m20.815670276s) [upload=187687]: Next returned nil segment Mar 18 19:34:17.800018306 (195h23m20.815687917s) [upload=187687]: Waiting for error group managing segments... Mar 18 19:34:17.946303744 (195h23m20.961973334s) [noise=true , address=45.91.116.13:4179 , piece_id=RMEDLCAM3SED327FL2ENLJFAYUX6WB3BMU364UB3RHE2YY7NJ42A, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:17.958150371 (195h23m20.973819975s) [noise=true , address=77.90.31.11:4575 , piece_id=P6B3YXSQQ4YRORDLS5POQY4ZCWPNTC554UEZD7PAKHFWJ2QGTENQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:17.974046250 (195h23m20.989715855s) [noise=true , address=79.127.219.38:4851 , piece_id=F4EGHAVII6L4EY626YFAAPHTUD22HMAJCZZXJQDCQ5D7GTICUOTA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:17.978120808 (195h23m20.993790412s) [noise=true , address=45.91.116.14:4198 , piece_id=P4ATZCI44XZCYV4ZHHEWRIBZTI25N2YPLUQACKQSFHU5ZIDNAM4A, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:17.981186396 (195h23m20.996855996s) [noise=true , address=77.90.31.14:4680 , piece_id=ZQM2LQ6DIW4I2KBPANJG6UXDEX54EQWCT74HYPUFFZJ3NYV4FWBA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.301445669 (195h23m21.317115272s) [noise=true , address=167.17.72.71:4816 , piece_id=N2SOF6MSXA2NYKTPSVDSMDSNKLZYYAMTK3TARDOB6QM3S24MMROA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.331480464 (195h23m21.347150077s) [noise=true , address=167.17.72.69:4740 , piece_id=6Q5JOGHJV5NETPZ53INLNDL64KUTOH6FKQNX23UOQ2TBBPSSDYJQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.480437207 (195h23m21.49610682s) [noise=true , address=193.149.164.68:4061 , piece_id=I2YF5IQTOS3KSWL3GBSSBOCR4J4V7BT4PN3VWZGUSGMBAPILSHPQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.506341021 (195h23m21.522010628s) [noise=true , address=193.149.164.67:4029 , piece_id=AORLLFAMQKQ6WH3IIXYCS5R5EJW2CC7X4K5SWCXNXB27T6JFZWZA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.682660820 (195h23m21.698330443s) [noise=true , address=79.127.205.237:4237 , piece_id=2TFJA4FUPBQP56URZ3DVRC6BWDIGJOEPOBI6TXVSVEUW2SPVII2Q, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:35:16.852210743 (195h24m19.867880349s) [noise=true , address=145.79.227.33:4883 , piece_id=4KT3BZ5RLJU6VZ3JSPYJ5GV3TQPCMRMQDQQCLFMSSTTHXPQZ2Z6Q, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:12cx1avL4umDm2pzJS1rNZqeDZjZMWnMtNsyNTdMdkxhpa1ByUN, address:145.79.227.33:4883): protocol: rpc: dial tcp 145.79.227.33:4883: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.852525149 (195h24m19.868194752s) [noise=true , address=145.79.227.43:4176 , piece_id=NLWMKDHQ4P5GBCD6OWAL5DCN6U27BBARHMEY2TFEQ55CD7UN3LMA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:1Yi5AibNiCNV8vUb9MKgooHCw8Nr79m9HqTrkrYybNZmatg9PD, address:145.79.227.43:4176): protocol: rpc: dial tcp 145.79.227.43:4176: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.852540920 (195h24m19.868210529s) [noise=true , address=145.79.227.23:4580 , piece_id=TWT5DAXYGGEXAFP7EOXDBUJ3JFGDXCWAG46O2NMAAN2DZVRRNI4A, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:12569SWj8WmR8tWoBXk7XJK1KS6BZMcYHrQjPKGfhj21xFPQ1W3, address:145.79.227.23:4580): protocol: rpc: dial tcp 145.79.227.23:4580: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.852795830 (195h24m19.868465442s) [noise=true , address=145.79.227.22:4519 , piece_id=N5H4SKH46OFD47RN3TVB2OSWVCPY7KRMXHYXP4FO27JW54IQN7RQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:1p5DYJfayCq68BQGnSEAoZCri8TWpST5WNcx5QFvzcxLyFamnj, address:145.79.227.22:4519): protocol: rpc: dial tcp 145.79.227.22:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.852913053 (195h24m19.868582662s) [noise=true , address=145.79.227.24:4603 , piece_id=GXKKSMCUK4ZI3IFHB5XVEVVGGDBW3VDZNS3H72BZJR3NOL2ABIVQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:145.79.227.24:4603): protocol: rpc: dial tcp 145.79.227.24:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.853081685 (195h24m19.868751288s) [noise=true , address=145.79.227.49:4358 , piece_id=NGBDKRHG4KJJQJIWWSLPZTFVDYWHAQJ73W55ZW5MPBRIC4LDZTOQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:1dwCNEcA9o51ZN3UaN3Frxdawwf8DXBj86iQMywjW8XP4ESmw1, address:145.79.227.49:4358): protocol: rpc: dial tcp 145.79.227.49:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.853184716 (195h24m19.868854329s) [noise=true , address=145.79.227.25:4645 , piece_id=FYSRVAFVZL424CYO6DSGFGWJLNZW6RWHNWVX6HSPTDZSYVLPBYOA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:12NPZfT1KXFLcfPTjX8zLQ4gpagKYosJbvCTKu9sK7989hDjTPm, address:145.79.227.25:4645): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:35:16.853380516 (195h24m19.869050119s) [noise=true , address=145.79.227.28:4754 , piece_id=4OYL73BSJ7VQUHI6KXS47AYGX34SHXZPD6HWCCLUXZWOC7U7EBZQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:1HB1v3ZURPMSYsbHBjJs1neE7HQtLS1KmeLwbKK2sTKAZcXPhi, address:145.79.227.28:4754): protocol: rpc: dial tcp 145.79.227.28:4754: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.853414478 (195h24m19.86908408s) [seg_pos={0 0}, upload=187687]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Mar 18 19:35:16.961105067 (195h24m19.976774674s) [upload=187687]: Done uploading segments... Mar 18 19:35:16.961154576 (195h24m19.976824177s) [upload=187687]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 3.26552481s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.111014676s trace: https://jaeger.dev.storj.io/trace/1dbbc2c03b82519a FAIL (2026-03-18 19:35:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 682742s) [noise=true , address=45.91.116.14:4198 , piece_id=P4ATZCI44XZCYV4ZHHEWRIBZTI25N2YPLUQACKQSFHU5ZIDNAM4A, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.243123447 (195h23m20.258793057s) [noise=true , address=145.79.227.25:4645 , piece_id=FYSRVAFVZL424CYO6DSGFGWJLNZW6RWHNWVX6HSPTDZSYVLPBYOA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.244312753 (195h23m20.259982353s) [noise=true , address=145.79.227.23:4580 , piece_id=TWT5DAXYGGEXAFP7EOXDBUJ3JFGDXCWAG46O2NMAAN2DZVRRNI4A, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.244446784 (195h23m20.260116392s) [noise=true , address=45.91.116.13:4179 , piece_id=RMEDLCAM3SED327FL2ENLJFAYUX6WB3BMU364UB3RHE2YY7NJ42A, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.244846632 (195h23m20.260516238s) [noise=true , address=145.79.227.22:4519 , piece_id=N5H4SKH46OFD47RN3TVB2OSWVCPY7KRMXHYXP4FO27JW54IQN7RQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.244928519 (195h23m20.260598129s) [noise=true , address=193.149.164.67:4029 , piece_id=AORLLFAMQKQ6WH3IIXYCS5R5EJW2CC7X4K5SWCXNXB27T6JFZWZA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.245353921 (195h23m20.26102353s) [noise=true , address=167.17.72.71:4816 , piece_id=N2SOF6MSXA2NYKTPSVDSMDSNKLZYYAMTK3TARDOB6QM3S24MMROA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.245385841 (195h23m20.261055658s) [noise=true , address=145.79.227.49:4358 , piece_id=NGBDKRHG4KJJQJIWWSLPZTFVDYWHAQJ73W55ZW5MPBRIC4LDZTOQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.246153076 (195h23m20.261822681s) [noise=true , address=77.90.31.11:4575 , piece_id=P6B3YXSQQ4YRORDLS5POQY4ZCWPNTC554UEZD7PAKHFWJ2QGTENQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.246634250 (195h23m20.262303852s) [noise=true , address=145.79.227.24:4603 , piece_id=GXKKSMCUK4ZI3IFHB5XVEVVGGDBW3VDZNS3H72BZJR3NOL2ABIVQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.246676628 (195h23m20.262346237s) [noise=true , address=167.17.72.69:4740 , piece_id=6Q5JOGHJV5NETPZ53INLNDL64KUTOH6FKQNX23UOQ2TBBPSSDYJQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.247101368 (195h23m20.26277097s) [noise=true , address=193.149.164.68:4061 , piece_id=I2YF5IQTOS3KSWL3GBSSBOCR4J4V7BT4PN3VWZGUSGMBAPILSHPQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.247375193 (195h23m20.263044797s) [noise=true , address=79.127.219.38:4851 , piece_id=F4EGHAVII6L4EY626YFAAPHTUD22HMAJCZZXJQDCQ5D7GTICUOTA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.247656193 (195h23m20.263325783s) [noise=true , address=145.79.227.33:4883 , piece_id=4KT3BZ5RLJU6VZ3JSPYJ5GV3TQPCMRMQDQQCLFMSSTTHXPQZ2Z6Q, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.247789006 (195h23m20.263458598s) [noise=true , address=79.127.205.237:4237 , piece_id=2TFJA4FUPBQP56URZ3DVRC6BWDIGJOEPOBI6TXVSVEUW2SPVII2Q, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.248224855 (195h23m20.263894465s) [noise=true , address=145.79.227.28:4754 , piece_id=4OYL73BSJ7VQUHI6KXS47AYGX34SHXZPD6HWCCLUXZWOC7U7EBZQ, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.248582486 (195h23m20.264252076s) [noise=true , address=145.79.227.43:4176 , piece_id=NLWMKDHQ4P5GBCD6OWAL5DCN6U27BBARHMEY2TFEQ55CD7UN3LMA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.248836068 (195h23m20.264505661s) [noise=true , address=77.90.31.14:4680 , piece_id=ZQM2LQ6DIW4I2KBPANJG6UXDEX54EQWCT74HYPUFFZJ3NYV4FWBA, seg_pos={0 0}, upload=187687]: Uploading piece... Mar 18 19:34:17.800000668 (195h23m20.815670276s) [upload=187687]: Next returned nil segment Mar 18 19:34:17.800018306 (195h23m20.815687917s) [upload=187687]: Waiting for error group managing segments... Mar 18 19:34:17.946303744 (195h23m20.961973334s) [noise=true , address=45.91.116.13:4179 , piece_id=RMEDLCAM3SED327FL2ENLJFAYUX6WB3BMU364UB3RHE2YY7NJ42A, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:17.958150371 (195h23m20.973819975s) [noise=true , address=77.90.31.11:4575 , piece_id=P6B3YXSQQ4YRORDLS5POQY4ZCWPNTC554UEZD7PAKHFWJ2QGTENQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:17.974046250 (195h23m20.989715855s) [noise=true , address=79.127.219.38:4851 , piece_id=F4EGHAVII6L4EY626YFAAPHTUD22HMAJCZZXJQDCQ5D7GTICUOTA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:17.978120808 (195h23m20.993790412s) [noise=true , address=45.91.116.14:4198 , piece_id=P4ATZCI44XZCYV4ZHHEWRIBZTI25N2YPLUQACKQSFHU5ZIDNAM4A, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:17.981186396 (195h23m20.996855996s) [noise=true , address=77.90.31.14:4680 , piece_id=ZQM2LQ6DIW4I2KBPANJG6UXDEX54EQWCT74HYPUFFZJ3NYV4FWBA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.301445669 (195h23m21.317115272s) [noise=true , address=167.17.72.71:4816 , piece_id=N2SOF6MSXA2NYKTPSVDSMDSNKLZYYAMTK3TARDOB6QM3S24MMROA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.331480464 (195h23m21.347150077s) [noise=true , address=167.17.72.69:4740 , piece_id=6Q5JOGHJV5NETPZ53INLNDL64KUTOH6FKQNX23UOQ2TBBPSSDYJQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.480437207 (195h23m21.49610682s) [noise=true , address=193.149.164.68:4061 , piece_id=I2YF5IQTOS3KSWL3GBSSBOCR4J4V7BT4PN3VWZGUSGMBAPILSHPQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.506341021 (195h23m21.522010628s) [noise=true , address=193.149.164.67:4029 , piece_id=AORLLFAMQKQ6WH3IIXYCS5R5EJW2CC7X4K5SWCXNXB27T6JFZWZA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:34:18.682660820 (195h23m21.698330443s) [noise=true , address=79.127.205.237:4237 , piece_id=2TFJA4FUPBQP56URZ3DVRC6BWDIGJOEPOBI6TXVSVEUW2SPVII2Q, seg_pos={0 0}, upload=187687]: Done uploading piece. err: <nil> Mar 18 19:35:16.852210743 (195h24m19.867880349s) [noise=true , address=145.79.227.33:4883 , piece_id=4KT3BZ5RLJU6VZ3JSPYJ5GV3TQPCMRMQDQQCLFMSSTTHXPQZ2Z6Q, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:12cx1avL4umDm2pzJS1rNZqeDZjZMWnMtNsyNTdMdkxhpa1ByUN, address:145.79.227.33:4883): protocol: rpc: dial tcp 145.79.227.33:4883: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.852525149 (195h24m19.868194752s) [noise=true , address=145.79.227.43:4176 , piece_id=NLWMKDHQ4P5GBCD6OWAL5DCN6U27BBARHMEY2TFEQ55CD7UN3LMA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:1Yi5AibNiCNV8vUb9MKgooHCw8Nr79m9HqTrkrYybNZmatg9PD, address:145.79.227.43:4176): protocol: rpc: dial tcp 145.79.227.43:4176: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.852540920 (195h24m19.868210529s) [noise=true , address=145.79.227.23:4580 , piece_id=TWT5DAXYGGEXAFP7EOXDBUJ3JFGDXCWAG46O2NMAAN2DZVRRNI4A, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:12569SWj8WmR8tWoBXk7XJK1KS6BZMcYHrQjPKGfhj21xFPQ1W3, address:145.79.227.23:4580): protocol: rpc: dial tcp 145.79.227.23:4580: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.852795830 (195h24m19.868465442s) [noise=true , address=145.79.227.22:4519 , piece_id=N5H4SKH46OFD47RN3TVB2OSWVCPY7KRMXHYXP4FO27JW54IQN7RQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:1p5DYJfayCq68BQGnSEAoZCri8TWpST5WNcx5QFvzcxLyFamnj, address:145.79.227.22:4519): protocol: rpc: dial tcp 145.79.227.22:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.852913053 (195h24m19.868582662s) [noise=true , address=145.79.227.24:4603 , piece_id=GXKKSMCUK4ZI3IFHB5XVEVVGGDBW3VDZNS3H72BZJR3NOL2ABIVQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:12ASAeTMBu4sY1GuZiSuwTXmhPksA1Te2BVZdcoE9QvpLAmcQX3, address:145.79.227.24:4603): protocol: rpc: dial tcp 145.79.227.24:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.853081685 (195h24m19.868751288s) [noise=true , address=145.79.227.49:4358 , piece_id=NGBDKRHG4KJJQJIWWSLPZTFVDYWHAQJ73W55ZW5MPBRIC4LDZTOQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:1dwCNEcA9o51ZN3UaN3Frxdawwf8DXBj86iQMywjW8XP4ESmw1, address:145.79.227.49:4358): protocol: rpc: dial tcp 145.79.227.49:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.853184716 (195h24m19.868854329s) [noise=true , address=145.79.227.25:4645 , piece_id=FYSRVAFVZL424CYO6DSGFGWJLNZW6RWHNWVX6HSPTDZSYVLPBYOA, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:12NPZfT1KXFLcfPTjX8zLQ4gpagKYosJbvCTKu9sK7989hDjTPm, address:145.79.227.25:4645): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:35:16.853380516 (195h24m19.869050119s) [noise=true , address=145.79.227.28:4754 , piece_id=4OYL73BSJ7VQUHI6KXS47AYGX34SHXZPD6HWCCLUXZWOC7U7EBZQ, seg_pos={0 0}, upload=187687]: Done uploading piece. err: ecclient: upload failed (node:1HB1v3ZURPMSYsbHBjJs1neE7HQtLS1KmeLwbKK2sTKAZcXPhi, address:145.79.227.28:4754): protocol: rpc: dial tcp 145.79.227.28:4754: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:16.853414478 (195h24m19.86908408s) [seg_pos={0 0}, upload=187687]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Mar 18 19:35:16.961105067 (195h24m19.976774674s) [upload=187687]: Done uploading segments... Mar 18 19:35:16.961154576 (195h24m19.976824177s) [upload=187687]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.111042375s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.110425771s trace: https://jaeger.dev.storj.io/trace/23fa351baecea88c FAIL (2026-03-18 19:35:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 736551 (195h23m43.271406162s) [upload=187693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:34:40.367441447 (195h23m43.383111049s) [seg_pos={0 0}, upload=187693]: Begin upload segment... Mar 18 19:34:40.367523345 (195h23m43.383192948s) [seg_pos={0 0}, upload=187693]: Done begin upload segment. Mar 18 19:34:40.367793471 (195h23m43.383463071s) [noise=true , address=145.79.227.46:4602 , piece_id=UFFPO23JERIW5QLZVQTIM5ZDVWDTSBGITXOIJV56EQLJG2WJOEHQ, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.368597077 (195h23m43.384266667s) [noise=true , address=145.79.227.9:4369 , piece_id=RKENU3XIWWA57PIKWZMKBPUAUNVUCRNIB2AEM5STBHAS5L6COKKA, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.369190307 (195h23m43.384859893s) [noise=true , address=77.90.31.14:4665 , piece_id=DFAEWQV2CEBERTUZQTGHD3WQTCRJWYEDCMKQQLWHA6TBXJGY7VPA, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.370581103 (195h23m43.386250692s) [noise=true , address=145.79.227.2:4119 , piece_id=WF75ACG4CHSKJXDPGHW22SF5HPZV6YLDVZEQ2IFTCXLCP74OW7OQ, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.371316326 (195h23m43.386985915s) [noise=true , address=79.127.219.41:4925 , piece_id=MAHYAFLELMA75YYI7RLE27CWOJVDF7IQH3OEKMAPLJLO5CDBRUVA, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.371687442 (195h23m43.387357034s) [noise=true , address=145.79.227.44:4193 , piece_id=RXXBMIKNIMMFFAJLAHLFZ42D22YBXQJ6QCB4J3QHVHC5MO5KBKLQ, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.372176266 (195h23m43.387845861s) [noise=true , address=145.79.227.52:4449 , piece_id=QGL7T5FDJ5ZXNXB34S2VA4QYCZN7WEU72CTX5GA5HWZDPHQ763UQ, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.372671406 (195h23m43.388341007s) [noise=true , address=145.79.227.6:4262 , piece_id=BZ4RSCVRIAFX54Z3IC4TS63YETBLJBXFF3AERHU6TEBEYL2LLRGQ, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.372861258 (195h23m43.388530851s) [noise=true , address=77.90.31.16:4738 , piece_id=YU4MREFUXKRRZJTQZN4LN3QBYPW4TYX2KVT7PW3PYEEQWHOJC44A, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.373380070 (195h23m43.389049663s) [noise=true , address=77.90.31.11:4557 , piece_id=FEVAIY432VZPGGLFFSPK63VEXR6WX6ZG3V2UAHQMVOTBW4OCMZ4A, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.374326113 (195h23m43.389995707s) [noise=true , address=77.90.31.17:4773 , piece_id=OAT6WIHENXRQXEGXAMRJQNTY7SFW2XA32QJIAGVLYUUCUZNKAPTQ, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.374795786 (195h23m43.390465378s) [noise=true , address=145.79.227.51:4412 , piece_id=Q6COPAXCLWWIARGTYMWNDLOYE4SUUWBSD4FH2ADGZSPYGXUUCTRA, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.374998042 (195h23m43.390667635s) [noise=true , address=45.91.116.12:4226 , piece_id=HH3HXFECNCTMS3APL6GQAZ27LYJDCZ53BI6C4KFZXOQGW44JQU2Q, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.375453713 (195h23m43.391123307s) [noise=true , address=145.79.227.29:4946 , piece_id=XFV5FIT7GKGHHUOSGIDMSRMV6AHQGO3DWOWHI6M5EJ347EI6OQVA, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.376486019 (195h23m43.392155611s) [noise=true , address=145.79.227.39:4017 , piece_id=YXL6UTWIHD5O44SCAIYQYFSOFMNPGJXHHGOZ3SEBF3RM7AGF5ETQ, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.378064781 (195h23m43.393734379s) [noise=true , address=145.79.227.47:4265 , piece_id=G3TL4HEUNKYGJRHKZ4MJOJMCTHZMPX56NJ3G34BM36WC3ZD73TKQ, seg_pos={0 0}, upload=187693]: Uploading piece... Mar 18 19:34:40.969257086 (195h23m43.984926693s) [upload=187693]: Next returned nil segment Mar 18 19:34:40.969275654 (195h23m43.984945256s) [upload=187693]: Waiting for error group managing segments... Mar 18 19:34:41.109460965 (195h23m44.125130565s) [noise=true , address=77.90.31.11:4557 , piece_id=FEVAIY432VZPGGLFFSPK63VEXR6WX6ZG3V2UAHQMVOTBW4OCMZ4A, seg_pos={0 0}, upload=187693]: Done uploading piece. err: <nil> Mar 18 19:34:41.116707587 (195h23m44.132377192s) [noise=true , address=77.90.31.14:4665 , piece_id=DFAEWQV2CEBERTUZQTGHD3WQTCRJWYEDCMKQQLWHA6TBXJGY7VPA, seg_pos={0 0}, upload=187693]: Done uploading piece. err: <nil> Mar 18 19:34:41.124133976 (195h23m44.139803576s) [noise=true , address=45.91.116.12:4226 , piece_id=HH3HXFECNCTMS3APL6GQAZ27LYJDCZ53BI6C4KFZXOQGW44JQU2Q, seg_pos={0 0}, upload=187693]: Done uploading piece. err: <nil> Mar 18 19:34:41.127451994 (195h23m44.143121595s) [noise=true , address=77.90.31.16:4738 , piece_id=YU4MREFUXKRRZJTQZN4LN3QBYPW4TYX2KVT7PW3PYEEQWHOJC44A, seg_pos={0 0}, upload=187693]: Done uploading piece. err: <nil> Mar 18 19:34:41.132328375 (195h23m44.14799798s) [noise=true , address=77.90.31.17:4773 , piece_id=OAT6WIHENXRQXEGXAMRJQNTY7SFW2XA32QJIAGVLYUUCUZNKAPTQ, seg_pos={0 0}, upload=187693]: Done uploading piece. err: <nil> Mar 18 19:34:41.177725239 (195h23m44.193394848s) [noise=true , address=79.127.219.41:4925 , piece_id=MAHYAFLELMA75YYI7RLE27CWOJVDF7IQH3OEKMAPLJLO5CDBRUVA, seg_pos={0 0}, upload=187693]: Done uploading piece. err: <nil> Mar 18 19:35:09.962193449 (195h24m12.97786304s) [noise=true , address=145.79.227.9:4369 , piece_id=RKENU3XIWWA57PIKWZMKBPUAUNVUCRNIB2AEM5STBHAS5L6COKKA, seg_pos={0 0}, upload=187693]: Done uploading piece. err: ecclient: upload failed (node:1nwygPHyRJEZkjCoSZJTVhXDKLHc7AonVMzKMht6ppwZwhcGCT, address:145.79.227.9:4369): protocol: rpc: dial tcp 145.79.227.9:4369: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:09.962477013 (195h24m12.978146621s) [noise=true , address=145.79.227.2:4119 , piece_id=WF75ACG4CHSKJXDPGHW22SF5HPZV6YLDVZEQ2IFTCXLCP74OW7OQ, seg_pos={0 0}, upload=187693]: Done uploading piece. err: ecclient: upload failed (node:121xNAnkm9HZXUszp26UmWTsuhp3igyeDVea6gSCdtrQUQDVHpW, address:145.79.227.2:4119): protocol: rpc: dial tcp 145.79.227.2:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:09.962610120 (195h24m12.978279742s) [noise=true , address=145.79.227.52:4449 , piece_id=QGL7T5FDJ5ZXNXB34S2VA4QYCZN7WEU72CTX5GA5HWZDPHQ763UQ, seg_pos={0 0}, upload=187693]: Done uploading piece. err: ecclient: upload failed (node:12f14pHDEEamEnHmeZCe38rjaE6EYWYkGMPrZMsbSDmApN5rckq, address:145.79.227.52:4449): protocol: rpc: dial tcp 145.79.227.52:4449: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:09.962782754 (195h24m12.978452364s) [noise=true , address=145.79.227.39:4017 , piece_id=YXL6UTWIHD5O44SCAIYQYFSOFMNPGJXHHGOZ3SEBF3RM7AGF5ETQ, seg_pos={0 0}, upload=187693]: Done uploading piece. err: ecclient: upload failed (node:12ocB4w2b2owjyWVCcq59Ymo2prJfSFmK5YogQDjGBroF11Tegk, address:145.79.227.39:4017): protocol: rpc: dial tcp 145.79.227.39:4017: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:09.963001847 (195h24m12.978671459s) [noise=true , address=145.79.227.47:4265 , piece_id=G3TL4HEUNKYGJRHKZ4MJOJMCTHZMPX56NJ3G34BM36WC3ZD73TKQ, seg_pos={0 0}, upload=187693]: Done uploading piece. err: ecclient: upload failed (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, address:145.79.227.47:4265): protocol: rpc: dial tcp 145.79.227.47:4265: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:09.963076399 (195h24m12.978746002s) [noise=true , address=145.79.227.44:4193 , piece_id=RXXBMIKNIMMFFAJLAHLFZ42D22YBXQJ6QCB4J3QHVHC5MO5KBKLQ, seg_pos={0 0}, upload=187693]: Done uploading piece. err: ecclient: upload failed (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, address:145.79.227.44:4193): protocol: rpc: dial tcp 145.79.227.44:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:09.963285988 (195h24m12.978955591s) [noise=true , address=145.79.227.6:4262 , piece_id=BZ4RSCVRIAFX54Z3IC4TS63YETBLJBXFF3AERHU6TEBEYL2LLRGQ, seg_pos={0 0}, upload=187693]: Done uploading piece. err: ecclient: upload failed (node:12nPw17QDTyrgDBBNKuunFWuDqPV5Rd2GpM7Loc6Ws9vAAZz22s, address:145.79.227.6:4262): protocol: rpc: dial tcp 145.79.227.6:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:09.963349629 (195h24m12.979019243s) [noise=true , address=145.79.227.51:4412 , piece_id=Q6COPAXCLWWIARGTYMWNDLOYE4SUUWBSD4FH2ADGZSPYGXUUCTRA, seg_pos={0 0}, upload=187693]: Done uploading piece. err: ecclient: upload failed (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS, address:145.79.227.51:4412): protocol: rpc: dial tcp 145.79.227.51:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:09.963820148 (195h24m12.979489759s) [noise=true , address=145.79.227.29:4946 , piece_id=XFV5FIT7GKGHHUOSGIDMSRMV6AHQGO3DWOWHI6M5EJ347EI6OQVA, seg_pos={0 0}, upload=187693]: Done uploading piece. err: ecclient: upload failed (node:12v9Q7aigjWPNXyshHmQt9YKrDu5af1vd1TxkDeHeKixSrWgrLX, address:145.79.227.29:4946): protocol: rpc: dial tcp 145.79.227.29:4946: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:09.964235420 (195h24m12.979905032s) [noise=true , address=145.79.227.46:4602 , piece_id=UFFPO23JERIW5QLZVQTIM5ZDVWDTSBGITXOIJV56EQLJG2WJOEHQ, seg_pos={0 0}, upload=187693]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: rpc: dial tcp 145.79.227.46:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:09.964285290 (195h24m12.979954894s) [seg_pos={0 0}, upload=187693]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Mar 18 19:35:10.069689830 (195h24m13.085359429s) [upload=187693]: Done uploading segments... Mar 18 19:35:10.069744604 (195h24m13.085414205s) [upload=187693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 30.11044186s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........................................................XX] duration: 5.001498507s trace: https://jaeger.dev.storj.io/trace/25b5ba2bce3d7bbb FAIL (2026-03-18 19:35:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001528249s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.000676844s trace: https://jaeger.dev.storj.io/trace/17e71f217c3e99f7 FAIL (2026-03-18 19:34:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000696935s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.10720109s trace: https://jaeger.dev.storj.io/trace/5d0279874f0b9f38 FAIL (2026-03-18 19:34:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== , seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.100644425 (195h23m13.116314036s) [noise=true , address=145.79.227.51:4412 , piece_id=INPSWZTGL435C5IVZ2E7MA3WXY6E5WHFRO7WGJZ2YNO7LCVU6WDA, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.101133036 (195h23m13.116802646s) [noise=true , address=145.79.227.2:4118 , piece_id=T7XKGKAXQIYVRMQ2XN3I7YFNR2DRC3KEYPJ2KURV2OL6RA7TRIVA, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.101447996 (195h23m13.117117608s) [noise=true , address=145.79.227.50:4373 , piece_id=IZOU2LTVXCPUOETUR3UE4DCHGVSVUPSMGVBAV2FHTAULGTKABSQA, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.101631379 (195h23m13.117300989s) [noise=true , address=145.79.227.16:4913 , piece_id=NBVMESKIKN3SCYRG37U3PS7PMROVUNL3JPSWPQCPJNE5FHZMU4JQ, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.102096421 (195h23m13.117766022s) [noise=true , address=145.79.227.42:4121 , piece_id=73ZSSBIKBGR6KJQU7KSUJXQJMLLBP4V5IO3C26X72THA4SDOTV6Q, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.102327141 (195h23m13.117996751s) [noise=true , address=77.90.31.12:4590 , piece_id=QRWAGR7M6IRUNCORMRSRMQHYHFGIHAICT6ZK4JCH74UNUKW4VMNQ, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.102617110 (195h23m13.118286717s) [noise=true , address=145.79.227.28:4728 , piece_id=OLPYFJARDGPGUNZ76XUSTGSKFLXJBXPC6CDT34AC6DCO6Q4CJTMA, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.102761897 (195h23m13.118431507s) [noise=true , address=145.79.227.7:4297 , piece_id=EDQZER4QN4DSBZMZT3IIQVG6JLCAGB4LFTTKFC6BA3LHYJFNJQCA, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.102815315 (195h23m13.118484922s) [noise=true , address=145.79.227.41:4088 , piece_id=MTJ756UPEOYQHG3MZG3ENQBTCGQGFZPUQ7OLWOT5LPG2SHOMFQOA, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.103273820 (195h23m13.118943437s) [noise=true , address=145.79.227.3:4149 , piece_id=C3TLE3APFQNQFSEZJF7O7C7EQHRVTD2QEWOUN4X3L7HGLY5CPQQA, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.103310204 (195h23m13.118979815s) [noise=true , address=79.127.219.38:4822 , piece_id=7B5KG24TB3A2A6HJZTKO5J7J37YY5FXCJHLOVNLOAWLMMN4PTZ2Q, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.103799534 (195h23m13.119469142s) [noise=true , address=145.79.227.5:4224 , piece_id=W3BLLAAMI54MODMHVOXQYZ7UMJWYHKQDEO7ESJRTPAAJQUFJ4CVA, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.104281120 (195h23m13.119950732s) [noise=true , address=145.79.227.4:4191 , piece_id=QVAOLXJVCBB7CHBK4GOASGYXCJVPULJQ3MFPKFESPGFYVNFJJAEA, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.104835215 (195h23m13.120504831s) [noise=true , address=45.91.116.5:4298 , piece_id=PS37Q2VP33ZTDAJWU2TDGEREABW53D6R75WVSP3TCQROPA2BGIEA, seg_pos={0 0}, upload=187683]: Uploading piece... Mar 18 19:34:10.712893069 (195h23m13.728562662s) [upload=187683]: Next returned nil segment Mar 18 19:34:10.712908272 (195h23m13.728577863s) [upload=187683]: Waiting for error group managing segments... Mar 18 19:34:10.863479381 (195h23m13.879148971s) [noise=true , address=77.90.31.12:4590 , piece_id=QRWAGR7M6IRUNCORMRSRMQHYHFGIHAICT6ZK4JCH74UNUKW4VMNQ, seg_pos={0 0}, upload=187683]: Done uploading piece. err: <nil> Mar 18 19:34:10.867544190 (195h23m13.883213803s) [noise=true , address=45.91.116.5:4298 , piece_id=PS37Q2VP33ZTDAJWU2TDGEREABW53D6R75WVSP3TCQROPA2BGIEA, seg_pos={0 0}, upload=187683]: Done uploading piece. err: <nil> Mar 18 19:34:10.906134406 (195h23m13.921804049s) [noise=true , address=79.127.219.38:4822 , piece_id=7B5KG24TB3A2A6HJZTKO5J7J37YY5FXCJHLOVNLOAWLMMN4PTZ2Q, seg_pos={0 0}, upload=187683]: Done uploading piece. err: <nil> Mar 18 19:34:39.702563298 (195h23m42.718232908s) [noise=true , address=145.79.227.2:4118 , piece_id=T7XKGKAXQIYVRMQ2XN3I7YFNR2DRC3KEYPJ2KURV2OL6RA7TRIVA, seg_pos={0 0}, upload=187683]: Done uploading piece. err: ecclient: upload failed (node:12ty5LHXwLmHC7nRPmVfj2wVEif6JQAQQ5Z5218vv1fKK5sD3zE, address:145.79.227.2:4118): protocol: rpc: dial tcp 145.79.227.2:4118: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:39.702959900 (195h23m42.718629511s) [noise=true , address=145.79.227.51:4412 , piece_id=INPSWZTGL435C5IVZ2E7MA3WXY6E5WHFRO7WGJZ2YNO7LCVU6WDA, seg_pos={0 0}, upload=187683]: Done uploading piece. err: ecclient: upload failed (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS, address:145.79.227.51:4412): protocol: rpc: dial tcp 145.79.227.51:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:39.703000452 (195h23m42.718670062s) [noise=true , address=145.79.227.28:4728 , piece_id=OLPYFJARDGPGUNZ76XUSTGSKFLXJBXPC6CDT34AC6DCO6Q4CJTMA, seg_pos={0 0}, upload=187683]: Done uploading piece. err: ecclient: upload failed (node:1YoLzpb4mMYZ8TsdxnMocekZZ1gLfTrdCM8mxTDj4tY6khZGvi, address:145.79.227.28:4728): protocol: rpc: dial tcp 145.79.227.28:4728: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:39.703294433 (195h23m42.718964042s) [noise=true , address=145.79.227.5:4224 , piece_id=W3BLLAAMI54MODMHVOXQYZ7UMJWYHKQDEO7ESJRTPAAJQUFJ4CVA, seg_pos={0 0}, upload=187683]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: rpc: dial tcp 145.79.227.5:4224: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:39.703397688 (195h23m42.719067297s) [noise=true , address=145.79.227.41:4088 , piece_id=MTJ756UPEOYQHG3MZG3ENQBTCGQGFZPUQ7OLWOT5LPG2SHOMFQOA, seg_pos={0 0}, upload=187683]: Done uploading piece. err: ecclient: upload failed (node:1284h6cPNVXJYMkSekGQVw8b7p89tkhpmgYLxTTYUDu2J7Wadoj, address:145.79.227.41:4088): protocol: rpc: dial tcp 145.79.227.41:4088: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:39.703599229 (195h23m42.719268838s) [noise=true , address=145.79.227.50:4373 , piece_id=IZOU2LTVXCPUOETUR3UE4DCHGVSVUPSMGVBAV2FHTAULGTKABSQA, seg_pos={0 0}, upload=187683]: Done uploading piece. err: ecclient: upload failed (node:12AQGSkJuNJDxoeqhChUenWxHwF5yK41UESCF63FQz3XvabXXfz, address:145.79.227.50:4373): protocol: rpc: dial tcp 145.79.227.50:4373: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:39.703671865 (195h23m42.719341475s) [noise=true , address=145.79.227.18:4009 , piece_id=T2KMPHF5OKNMG7OJBCYTJKIFE2G7ZO6HSB6RCWEJHRR7QUFLMYYA, seg_pos={0 0}, upload=187683]: Done uploading piece. err: ecclient: upload failed (node:12n33Xie5Hk1vaKc279HvGnL2QBEYsNzieL2grpnANJpy7Qu2wb, address:145.79.227.18:4009): protocol: rpc: dial tcp 145.79.227.18:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:39.703969648 (195h23m42.719639296s) [noise=true , address=145.79.227.16:4913 , piece_id=NBVMESKIKN3SCYRG37U3PS7PMROVUNL3JPSWPQCPJNE5FHZMU4JQ, seg_pos={0 0}, upload=187683]: Done uploading piece. err: ecclient: upload failed (node:15rBbsiT2s5kCYxgPgkai7dBHjfUu6mr4CXMWbYz7pFuJQCLnm, address:145.79.227.16:4913): protocol: rpc: dial tcp 145.79.227.16:4913: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:39.704031636 (195h23m42.719701246s) [noise=true , address=145.79.227.42:4121 , piece_id=73ZSSBIKBGR6KJQU7KSUJXQJMLLBP4V5IO3C26X72THA4SDOTV6Q, seg_pos={0 0}, upload=187683]: Done uploading piece. err: ecclient: upload failed (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh, address:145.79.227.42:4121): protocol: rpc: dial tcp 145.79.227.42:4121: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:39.704344074 (195h23m42.720013685s) [noise=true , address=145.79.227.4:4191 , piece_id=QVAOLXJVCBB7CHBK4GOASGYXCJVPULJQ3MFPKFESPGFYVNFJJAEA, seg_pos={0 0}, upload=187683]: Done uploading piece. err: ecclient: upload failed (node:1sVWXyKPnLTobAhk6GXZTTP67FbyY3XJ1e2svTiUEa3rrghd2q, address:145.79.227.4:4191): protocol: rpc: dial tcp 145.79.227.4:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:39.704372250 (195h23m42.720041863s) [noise=true , address=145.79.227.3:4149 , piece_id=C3TLE3APFQNQFSEZJF7O7C7EQHRVTD2QEWOUN4X3L7HGLY5CPQQA, seg_pos={0 0}, upload=187683]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:39.704631603 (195h23m42.720301217s) [noise=true , address=145.79.227.31:4803 , piece_id=MNWVQR5Q4ZER56DLEUUUNLZLQ4FJUN3M6UVDMJIXZ3K6W7P6B7TQ, seg_pos={0 0}, upload=187683]: Done uploading piece. err: ecclient: upload failed (node:12GFk3GkhB88UKQdZM8vZ9VLNuuYDrS5ThZjVAdbL2CtTQVMDEB, address:145.79.227.31:4803): protocol: rpc: dial tcp 145.79.227.31:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:39.704654704 (195h23m42.720324309s) [noise=true , address=145.79.227.7:4297 , piece_id=EDQZER4QN4DSBZMZT3IIQVG6JLCAGB4LFTTKFC6BA3LHYJFNJQCA, seg_pos={0 0}, upload=187683]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:39.704687617 (195h23m42.720357207s) [seg_pos={0 0}, upload=187683]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Mar 18 19:34:39.807895737 (195h23m42.823565338s) [upload=187683]: Done uploading segments... Mar 18 19:34:39.807985688 (195h23m42.823655288s) [upload=187683]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 30.107226654s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.001762564s trace: https://jaeger.dev.storj.io/trace/1cedfaa14bd74faa FAIL (2026-03-18 19:34:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001785679s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 1.591735114s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 19:05:40 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.591763292s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.X..........................................................] duration: 1.609757365s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 19:05:23 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.60986434s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..X.........................................................] duration: 1.556416726s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 19:04:26 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.55651467s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...X........................................................] duration: 2.161665571s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 19:03:34 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.161692156s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....X.......................................................] duration: 1.650030633s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 19:02:37 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.650057457s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....X......................................................] duration: 2.238449775s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 19:01:27 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.238484973s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......X.....................................................] duration: 1.724266629s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 19:00:25 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.724342902s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......X....................................................] duration: 1.598900534s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:59:29 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.59900894s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........X...................................................] duration: 1.628652866s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:59:01 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.628677337s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........X..................................................] duration: 1.639341845s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:58:27 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.639382015s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........X.................................................] duration: 1.674472262s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:58:02 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.674500467s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........X................................................] duration: 1.704261303s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:57:06 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.704291104s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............X...............................................] duration: 1.653152404s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:55:59 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.653180867s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............X..............................................] duration: 1.727198113s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:55:08 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.72722465s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............X.............................................] duration: 2.097196008s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:54:00 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.097222827s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............X............................................] duration: 1.687534901s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:53:34 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.687561546s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................X...........................................] duration: 1.728170197s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:52:39 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.728201625s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................X..........................................] duration: 1.597666314s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:51:26 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.597688848s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................X.........................................] duration: 1.702107792s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:50:16 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.702139527s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................X........................................] duration: 2.132709631s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:49:54 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.132756025s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................X.......................................] duration: 1.694761965s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:48:51 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.694795066s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................X......................................] duration: 1.603830019s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:47:55 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.603860506s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................X.....................................] duration: 1.602261166s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:47:24 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.602342802s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................X....................................] duration: 1.545116069s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:46:33 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.54515021s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................X...................................] duration: 1.587476443s trace: https://jaeger.dev.storj.io/trace/61fe779357ffe884 FAIL (2026-03-18 18:45:41 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.58750243s core uplink us-select-1 us1-placement-20

1
...
576
577
578
579
580
...
7090