Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 29 out of 60 is failed (threshold=3) [XXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX.......X.......] duration: 18.713859569s trace: https://jaeger.dev.storj.io/trace/7a32556c3299f77d FAIL (2026-04-18 02:31:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== pload=240474]: Done begin upload segment. Apr 18 02:25:51.051303397 (250h9m1.645414643s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052559793 (250h9m1.646671043s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052600155 (250h9m1.646711385s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052639728 (250h9m1.646750954s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053090817 (250h9m1.647202066s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053231933 (250h9m1.647343178s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053514417 (250h9m1.647625667s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053720653 (250h9m1.647831902s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053903088 (250h9m1.648014338s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054285730 (250h9m1.648396983s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054397651 (250h9m1.648508896s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054802296 (250h9m1.648913546s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054824840 (250h9m1.648936086s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055258766 (250h9m1.649370033s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055330406 (250h9m1.649441657s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055712191 (250h9m1.649823437s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055870389 (250h9m1.649981634s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.056357114 (250h9m1.650468364s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.142743972 (250h9m1.736855217s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1LV3A9EyGYTRvni2ENn6yYkBdxQiJFW6zLC9vikrezvz6F9dEX, address:145.79.227.38:4998): protocol: rpc: dial tcp 145.79.227.38:4998: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144049380 (250h9m1.738160628s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:122m1dk7cCwZfUKeq4BChF1UjXSt7swvt1L3HdiSybavXhLEcsb, address:145.79.227.31:4818): protocol: rpc: dial tcp 145.79.227.31:4818: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144916153 (250h9m1.739027412s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12eA7ZLTkgGyCRsnQSEC8oYM3y9oeB5MQ4FUueJNMb4FLZueNv2, address:145.79.227.49:4355): protocol: rpc: dial tcp 145.79.227.49:4355: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.632081968 (250h9m2.226193222s) [upload=240474]: Next returned nil segment Apr 18 02:25:51.632102736 (250h9m2.226213986s) [upload=240474]: Waiting for error group managing segments... Apr 18 02:25:51.779193064 (250h9m2.373304314s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.166785373 (250h9m2.760896615s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.184676686 (250h9m2.778787936s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.207075381 (250h9m2.801186622s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.299059914 (250h9m2.893171157s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.302165775 (250h9m2.89627702s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.465614425 (250h9m3.059725667s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.479819483 (250h9m3.073930722s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.533279958 (250h9m3.127391199s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.562759587 (250h9m3.156870833s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.742002496 (250h9m3.336113738s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:53.461038287 (250h9m4.055149536s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1UMjJdYFqUjDcAEamv69RrmqtBPT2wiAk585jBsrv6yxFESjNy, address:145.79.227.11:4743): protocol: expected piece hash; hashstore: write /app/config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/b8/log-00000000000227b8-00005052: no space left on device; EOF Apr 18 02:26:50.666039093 (250h10m1.260150343s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:145.79.227.33:4885): protocol: rpc: dial tcp 145.79.227.33:4885: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666101955 (250h10m1.260213203s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:15jMrmLPcRL8VeYUPzYNPhc2N7qZt4QgTsrSHoo4Zrt4cGrHHK, address:145.79.227.44:4220): protocol: rpc: dial tcp 145.79.227.44:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666324349 (250h10m1.260435599s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12bGaxiZQ1uHmbYMfJ273PHh1jni4AVtTjcBEM38ZFYm2j1rV5g, address:167.17.72.69:4734): protocol: write tcp 139.59.144.22:55810->167.17.72.69:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666367389 (250h10m1.260478826s) [seg_pos={0 0}, upload=240474]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Apr 18 02:26:50.767707041 (250h10m1.361818273s) [upload=240474]: Done uploading segments... Apr 18 02:26:50.767750393 (250h10m1.361861649s) [upload=240474]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 18.713975539s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 19 out of 60 is failed (threshold=6) [XXXXXXXXXX.X.XX..XX.X.....XX..X.............................] duration: 1.784301356s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:31:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.784331609s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.X..........X...............................................] duration: 3.039090931s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:31:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.039125924s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 30 out of 60 is failed (threshold=3) [XXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX.......X......] duration: 5.103259476s trace: https://jaeger.dev.storj.io/trace/7a32556c3299f77d FAIL (2026-04-18 02:31:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== pload=240474]: Done begin upload segment. Apr 18 02:25:51.051303397 (250h9m1.645414643s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052559793 (250h9m1.646671043s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052600155 (250h9m1.646711385s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052639728 (250h9m1.646750954s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053090817 (250h9m1.647202066s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053231933 (250h9m1.647343178s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053514417 (250h9m1.647625667s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053720653 (250h9m1.647831902s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053903088 (250h9m1.648014338s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054285730 (250h9m1.648396983s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054397651 (250h9m1.648508896s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054802296 (250h9m1.648913546s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054824840 (250h9m1.648936086s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055258766 (250h9m1.649370033s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055330406 (250h9m1.649441657s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055712191 (250h9m1.649823437s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055870389 (250h9m1.649981634s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.056357114 (250h9m1.650468364s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.142743972 (250h9m1.736855217s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1LV3A9EyGYTRvni2ENn6yYkBdxQiJFW6zLC9vikrezvz6F9dEX, address:145.79.227.38:4998): protocol: rpc: dial tcp 145.79.227.38:4998: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144049380 (250h9m1.738160628s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:122m1dk7cCwZfUKeq4BChF1UjXSt7swvt1L3HdiSybavXhLEcsb, address:145.79.227.31:4818): protocol: rpc: dial tcp 145.79.227.31:4818: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144916153 (250h9m1.739027412s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12eA7ZLTkgGyCRsnQSEC8oYM3y9oeB5MQ4FUueJNMb4FLZueNv2, address:145.79.227.49:4355): protocol: rpc: dial tcp 145.79.227.49:4355: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.632081968 (250h9m2.226193222s) [upload=240474]: Next returned nil segment Apr 18 02:25:51.632102736 (250h9m2.226213986s) [upload=240474]: Waiting for error group managing segments... Apr 18 02:25:51.779193064 (250h9m2.373304314s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.166785373 (250h9m2.760896615s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.184676686 (250h9m2.778787936s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.207075381 (250h9m2.801186622s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.299059914 (250h9m2.893171157s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.302165775 (250h9m2.89627702s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.465614425 (250h9m3.059725667s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.479819483 (250h9m3.073930722s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.533279958 (250h9m3.127391199s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.562759587 (250h9m3.156870833s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.742002496 (250h9m3.336113738s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:53.461038287 (250h9m4.055149536s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1UMjJdYFqUjDcAEamv69RrmqtBPT2wiAk585jBsrv6yxFESjNy, address:145.79.227.11:4743): protocol: expected piece hash; hashstore: write /app/config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/b8/log-00000000000227b8-00005052: no space left on device; EOF Apr 18 02:26:50.666039093 (250h10m1.260150343s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:145.79.227.33:4885): protocol: rpc: dial tcp 145.79.227.33:4885: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666101955 (250h10m1.260213203s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:15jMrmLPcRL8VeYUPzYNPhc2N7qZt4QgTsrSHoo4Zrt4cGrHHK, address:145.79.227.44:4220): protocol: rpc: dial tcp 145.79.227.44:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666324349 (250h10m1.260435599s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12bGaxiZQ1uHmbYMfJ273PHh1jni4AVtTjcBEM38ZFYm2j1rV5g, address:167.17.72.69:4734): protocol: write tcp 139.59.144.22:55810->167.17.72.69:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666367389 (250h10m1.260478826s) [seg_pos={0 0}, upload=240474]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Apr 18 02:26:50.767707041 (250h10m1.361818273s) [upload=240474]: Done uploading segments... Apr 18 02:26:50.767750393 (250h10m1.361861649s) [upload=240474]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.103298365s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 20 out of 60 is failed (threshold=6) [XXXXXXXXXXX.X.XX..XX.X.....XX..X............................] duration: 2.203657105s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:30:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.203685484s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..X..........X..............................................] duration: 3.644700364s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:30:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.644790299s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 31 out of 60 is failed (threshold=3) [XXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX.......X.....] duration: 3.647224453s trace: https://jaeger.dev.storj.io/trace/7a32556c3299f77d FAIL (2026-04-18 02:30:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== pload=240474]: Done begin upload segment. Apr 18 02:25:51.051303397 (250h9m1.645414643s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052559793 (250h9m1.646671043s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052600155 (250h9m1.646711385s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052639728 (250h9m1.646750954s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053090817 (250h9m1.647202066s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053231933 (250h9m1.647343178s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053514417 (250h9m1.647625667s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053720653 (250h9m1.647831902s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053903088 (250h9m1.648014338s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054285730 (250h9m1.648396983s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054397651 (250h9m1.648508896s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054802296 (250h9m1.648913546s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054824840 (250h9m1.648936086s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055258766 (250h9m1.649370033s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055330406 (250h9m1.649441657s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055712191 (250h9m1.649823437s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055870389 (250h9m1.649981634s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.056357114 (250h9m1.650468364s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.142743972 (250h9m1.736855217s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1LV3A9EyGYTRvni2ENn6yYkBdxQiJFW6zLC9vikrezvz6F9dEX, address:145.79.227.38:4998): protocol: rpc: dial tcp 145.79.227.38:4998: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144049380 (250h9m1.738160628s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:122m1dk7cCwZfUKeq4BChF1UjXSt7swvt1L3HdiSybavXhLEcsb, address:145.79.227.31:4818): protocol: rpc: dial tcp 145.79.227.31:4818: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144916153 (250h9m1.739027412s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12eA7ZLTkgGyCRsnQSEC8oYM3y9oeB5MQ4FUueJNMb4FLZueNv2, address:145.79.227.49:4355): protocol: rpc: dial tcp 145.79.227.49:4355: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.632081968 (250h9m2.226193222s) [upload=240474]: Next returned nil segment Apr 18 02:25:51.632102736 (250h9m2.226213986s) [upload=240474]: Waiting for error group managing segments... Apr 18 02:25:51.779193064 (250h9m2.373304314s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.166785373 (250h9m2.760896615s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.184676686 (250h9m2.778787936s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.207075381 (250h9m2.801186622s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.299059914 (250h9m2.893171157s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.302165775 (250h9m2.89627702s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.465614425 (250h9m3.059725667s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.479819483 (250h9m3.073930722s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.533279958 (250h9m3.127391199s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.562759587 (250h9m3.156870833s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.742002496 (250h9m3.336113738s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:53.461038287 (250h9m4.055149536s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1UMjJdYFqUjDcAEamv69RrmqtBPT2wiAk585jBsrv6yxFESjNy, address:145.79.227.11:4743): protocol: expected piece hash; hashstore: write /app/config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/b8/log-00000000000227b8-00005052: no space left on device; EOF Apr 18 02:26:50.666039093 (250h10m1.260150343s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:145.79.227.33:4885): protocol: rpc: dial tcp 145.79.227.33:4885: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666101955 (250h10m1.260213203s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:15jMrmLPcRL8VeYUPzYNPhc2N7qZt4QgTsrSHoo4Zrt4cGrHHK, address:145.79.227.44:4220): protocol: rpc: dial tcp 145.79.227.44:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666324349 (250h10m1.260435599s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12bGaxiZQ1uHmbYMfJ273PHh1jni4AVtTjcBEM38ZFYm2j1rV5g, address:167.17.72.69:4734): protocol: write tcp 139.59.144.22:55810->167.17.72.69:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666367389 (250h10m1.260478826s) [seg_pos={0 0}, upload=240474]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Apr 18 02:26:50.767707041 (250h10m1.361818273s) [upload=240474]: Done uploading segments... Apr 18 02:26:50.767750393 (250h10m1.361861649s) [upload=240474]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 3.64729487s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 21 out of 60 is failed (threshold=6) [XXXXXXXXXXXX.X.XX..XX.X.....XX..X...........................] duration: 2.273235487s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:29:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.273262019s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 32 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX.......X....] duration: 6.692178424s trace: https://jaeger.dev.storj.io/trace/7a32556c3299f77d FAIL (2026-04-18 02:29:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== pload=240474]: Done begin upload segment. Apr 18 02:25:51.051303397 (250h9m1.645414643s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052559793 (250h9m1.646671043s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052600155 (250h9m1.646711385s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052639728 (250h9m1.646750954s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053090817 (250h9m1.647202066s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053231933 (250h9m1.647343178s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053514417 (250h9m1.647625667s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053720653 (250h9m1.647831902s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053903088 (250h9m1.648014338s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054285730 (250h9m1.648396983s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054397651 (250h9m1.648508896s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054802296 (250h9m1.648913546s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054824840 (250h9m1.648936086s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055258766 (250h9m1.649370033s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055330406 (250h9m1.649441657s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055712191 (250h9m1.649823437s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055870389 (250h9m1.649981634s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.056357114 (250h9m1.650468364s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.142743972 (250h9m1.736855217s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1LV3A9EyGYTRvni2ENn6yYkBdxQiJFW6zLC9vikrezvz6F9dEX, address:145.79.227.38:4998): protocol: rpc: dial tcp 145.79.227.38:4998: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144049380 (250h9m1.738160628s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:122m1dk7cCwZfUKeq4BChF1UjXSt7swvt1L3HdiSybavXhLEcsb, address:145.79.227.31:4818): protocol: rpc: dial tcp 145.79.227.31:4818: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144916153 (250h9m1.739027412s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12eA7ZLTkgGyCRsnQSEC8oYM3y9oeB5MQ4FUueJNMb4FLZueNv2, address:145.79.227.49:4355): protocol: rpc: dial tcp 145.79.227.49:4355: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.632081968 (250h9m2.226193222s) [upload=240474]: Next returned nil segment Apr 18 02:25:51.632102736 (250h9m2.226213986s) [upload=240474]: Waiting for error group managing segments... Apr 18 02:25:51.779193064 (250h9m2.373304314s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.166785373 (250h9m2.760896615s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.184676686 (250h9m2.778787936s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.207075381 (250h9m2.801186622s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.299059914 (250h9m2.893171157s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.302165775 (250h9m2.89627702s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.465614425 (250h9m3.059725667s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.479819483 (250h9m3.073930722s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.533279958 (250h9m3.127391199s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.562759587 (250h9m3.156870833s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.742002496 (250h9m3.336113738s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:53.461038287 (250h9m4.055149536s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1UMjJdYFqUjDcAEamv69RrmqtBPT2wiAk585jBsrv6yxFESjNy, address:145.79.227.11:4743): protocol: expected piece hash; hashstore: write /app/config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/b8/log-00000000000227b8-00005052: no space left on device; EOF Apr 18 02:26:50.666039093 (250h10m1.260150343s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:145.79.227.33:4885): protocol: rpc: dial tcp 145.79.227.33:4885: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666101955 (250h10m1.260213203s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:15jMrmLPcRL8VeYUPzYNPhc2N7qZt4QgTsrSHoo4Zrt4cGrHHK, address:145.79.227.44:4220): protocol: rpc: dial tcp 145.79.227.44:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666324349 (250h10m1.260435599s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12bGaxiZQ1uHmbYMfJ273PHh1jni4AVtTjcBEM38ZFYm2j1rV5g, address:167.17.72.69:4734): protocol: write tcp 139.59.144.22:55810->167.17.72.69:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666367389 (250h10m1.260478826s) [seg_pos={0 0}, upload=240474]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Apr 18 02:26:50.767707041 (250h10m1.361818273s) [upload=240474]: Done uploading segments... Apr 18 02:26:50.767750393 (250h10m1.361861649s) [upload=240474]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 6.692310424s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...X..........X.............................................] duration: 3.054395754s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:29:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.054436437s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 33 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX.......X...] duration: 4.833905251s trace: https://jaeger.dev.storj.io/trace/7a32556c3299f77d FAIL (2026-04-18 02:28:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== pload=240474]: Done begin upload segment. Apr 18 02:25:51.051303397 (250h9m1.645414643s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052559793 (250h9m1.646671043s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052600155 (250h9m1.646711385s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052639728 (250h9m1.646750954s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053090817 (250h9m1.647202066s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053231933 (250h9m1.647343178s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053514417 (250h9m1.647625667s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053720653 (250h9m1.647831902s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053903088 (250h9m1.648014338s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054285730 (250h9m1.648396983s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054397651 (250h9m1.648508896s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054802296 (250h9m1.648913546s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054824840 (250h9m1.648936086s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055258766 (250h9m1.649370033s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055330406 (250h9m1.649441657s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055712191 (250h9m1.649823437s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055870389 (250h9m1.649981634s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.056357114 (250h9m1.650468364s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.142743972 (250h9m1.736855217s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1LV3A9EyGYTRvni2ENn6yYkBdxQiJFW6zLC9vikrezvz6F9dEX, address:145.79.227.38:4998): protocol: rpc: dial tcp 145.79.227.38:4998: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144049380 (250h9m1.738160628s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:122m1dk7cCwZfUKeq4BChF1UjXSt7swvt1L3HdiSybavXhLEcsb, address:145.79.227.31:4818): protocol: rpc: dial tcp 145.79.227.31:4818: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144916153 (250h9m1.739027412s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12eA7ZLTkgGyCRsnQSEC8oYM3y9oeB5MQ4FUueJNMb4FLZueNv2, address:145.79.227.49:4355): protocol: rpc: dial tcp 145.79.227.49:4355: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.632081968 (250h9m2.226193222s) [upload=240474]: Next returned nil segment Apr 18 02:25:51.632102736 (250h9m2.226213986s) [upload=240474]: Waiting for error group managing segments... Apr 18 02:25:51.779193064 (250h9m2.373304314s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.166785373 (250h9m2.760896615s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.184676686 (250h9m2.778787936s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.207075381 (250h9m2.801186622s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.299059914 (250h9m2.893171157s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.302165775 (250h9m2.89627702s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.465614425 (250h9m3.059725667s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.479819483 (250h9m3.073930722s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.533279958 (250h9m3.127391199s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.562759587 (250h9m3.156870833s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.742002496 (250h9m3.336113738s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:53.461038287 (250h9m4.055149536s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1UMjJdYFqUjDcAEamv69RrmqtBPT2wiAk585jBsrv6yxFESjNy, address:145.79.227.11:4743): protocol: expected piece hash; hashstore: write /app/config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/b8/log-00000000000227b8-00005052: no space left on device; EOF Apr 18 02:26:50.666039093 (250h10m1.260150343s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:145.79.227.33:4885): protocol: rpc: dial tcp 145.79.227.33:4885: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666101955 (250h10m1.260213203s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:15jMrmLPcRL8VeYUPzYNPhc2N7qZt4QgTsrSHoo4Zrt4cGrHHK, address:145.79.227.44:4220): protocol: rpc: dial tcp 145.79.227.44:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666324349 (250h10m1.260435599s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12bGaxiZQ1uHmbYMfJ273PHh1jni4AVtTjcBEM38ZFYm2j1rV5g, address:167.17.72.69:4734): protocol: write tcp 139.59.144.22:55810->167.17.72.69:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666367389 (250h10m1.260478826s) [seg_pos={0 0}, upload=240474]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Apr 18 02:26:50.767707041 (250h10m1.361818273s) [upload=240474]: Done uploading segments... Apr 18 02:26:50.767750393 (250h10m1.361861649s) [upload=240474]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 4.833926627s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 22 out of 60 is failed (threshold=6) [XXXXXXXXXXXXX.X.XX..XX.X.....XX..X..........................] duration: 2.310968379s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:28:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.310997385s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 23 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXX.X.XX..XX.X.....XX..X.........................] duration: 1.796199815s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:28:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.796217032s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....X..........X............................................] duration: 3.26712144s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:28:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.267163535s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 34 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX.......X..] duration: 3.725460545s trace: https://jaeger.dev.storj.io/trace/7a32556c3299f77d FAIL (2026-04-18 02:27:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== pload=240474]: Done begin upload segment. Apr 18 02:25:51.051303397 (250h9m1.645414643s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052559793 (250h9m1.646671043s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052600155 (250h9m1.646711385s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052639728 (250h9m1.646750954s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053090817 (250h9m1.647202066s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053231933 (250h9m1.647343178s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053514417 (250h9m1.647625667s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053720653 (250h9m1.647831902s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053903088 (250h9m1.648014338s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054285730 (250h9m1.648396983s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054397651 (250h9m1.648508896s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054802296 (250h9m1.648913546s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054824840 (250h9m1.648936086s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055258766 (250h9m1.649370033s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055330406 (250h9m1.649441657s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055712191 (250h9m1.649823437s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055870389 (250h9m1.649981634s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.056357114 (250h9m1.650468364s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.142743972 (250h9m1.736855217s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1LV3A9EyGYTRvni2ENn6yYkBdxQiJFW6zLC9vikrezvz6F9dEX, address:145.79.227.38:4998): protocol: rpc: dial tcp 145.79.227.38:4998: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144049380 (250h9m1.738160628s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:122m1dk7cCwZfUKeq4BChF1UjXSt7swvt1L3HdiSybavXhLEcsb, address:145.79.227.31:4818): protocol: rpc: dial tcp 145.79.227.31:4818: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144916153 (250h9m1.739027412s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12eA7ZLTkgGyCRsnQSEC8oYM3y9oeB5MQ4FUueJNMb4FLZueNv2, address:145.79.227.49:4355): protocol: rpc: dial tcp 145.79.227.49:4355: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.632081968 (250h9m2.226193222s) [upload=240474]: Next returned nil segment Apr 18 02:25:51.632102736 (250h9m2.226213986s) [upload=240474]: Waiting for error group managing segments... Apr 18 02:25:51.779193064 (250h9m2.373304314s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.166785373 (250h9m2.760896615s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.184676686 (250h9m2.778787936s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.207075381 (250h9m2.801186622s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.299059914 (250h9m2.893171157s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.302165775 (250h9m2.89627702s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.465614425 (250h9m3.059725667s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.479819483 (250h9m3.073930722s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.533279958 (250h9m3.127391199s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.562759587 (250h9m3.156870833s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.742002496 (250h9m3.336113738s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:53.461038287 (250h9m4.055149536s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1UMjJdYFqUjDcAEamv69RrmqtBPT2wiAk585jBsrv6yxFESjNy, address:145.79.227.11:4743): protocol: expected piece hash; hashstore: write /app/config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/b8/log-00000000000227b8-00005052: no space left on device; EOF Apr 18 02:26:50.666039093 (250h10m1.260150343s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:145.79.227.33:4885): protocol: rpc: dial tcp 145.79.227.33:4885: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666101955 (250h10m1.260213203s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:15jMrmLPcRL8VeYUPzYNPhc2N7qZt4QgTsrSHoo4Zrt4cGrHHK, address:145.79.227.44:4220): protocol: rpc: dial tcp 145.79.227.44:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666324349 (250h10m1.260435599s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12bGaxiZQ1uHmbYMfJ273PHh1jni4AVtTjcBEM38ZFYm2j1rV5g, address:167.17.72.69:4734): protocol: write tcp 139.59.144.22:55810->167.17.72.69:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666367389 (250h10m1.260478826s) [seg_pos={0 0}, upload=240474]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Apr 18 02:26:50.767707041 (250h10m1.361818273s) [upload=240474]: Done uploading segments... Apr 18 02:26:50.767750393 (250h10m1.361861649s) [upload=240474]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 3.725604699s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 24 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXX.X.XX..XX.X.....XX..X........................] duration: 1.707590752s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:27:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.707629346s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [X....X..........X...........................................] duration: 3.101098366s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:27:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.101125097s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 35 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX.......X.] duration: 4.186442795s trace: https://jaeger.dev.storj.io/trace/7a32556c3299f77d FAIL (2026-04-18 02:26:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== pload=240474]: Done begin upload segment. Apr 18 02:25:51.051303397 (250h9m1.645414643s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052559793 (250h9m1.646671043s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052600155 (250h9m1.646711385s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052639728 (250h9m1.646750954s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053090817 (250h9m1.647202066s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053231933 (250h9m1.647343178s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053514417 (250h9m1.647625667s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053720653 (250h9m1.647831902s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053903088 (250h9m1.648014338s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054285730 (250h9m1.648396983s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054397651 (250h9m1.648508896s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054802296 (250h9m1.648913546s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054824840 (250h9m1.648936086s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055258766 (250h9m1.649370033s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055330406 (250h9m1.649441657s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055712191 (250h9m1.649823437s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055870389 (250h9m1.649981634s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.056357114 (250h9m1.650468364s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.142743972 (250h9m1.736855217s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1LV3A9EyGYTRvni2ENn6yYkBdxQiJFW6zLC9vikrezvz6F9dEX, address:145.79.227.38:4998): protocol: rpc: dial tcp 145.79.227.38:4998: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144049380 (250h9m1.738160628s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:122m1dk7cCwZfUKeq4BChF1UjXSt7swvt1L3HdiSybavXhLEcsb, address:145.79.227.31:4818): protocol: rpc: dial tcp 145.79.227.31:4818: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144916153 (250h9m1.739027412s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12eA7ZLTkgGyCRsnQSEC8oYM3y9oeB5MQ4FUueJNMb4FLZueNv2, address:145.79.227.49:4355): protocol: rpc: dial tcp 145.79.227.49:4355: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.632081968 (250h9m2.226193222s) [upload=240474]: Next returned nil segment Apr 18 02:25:51.632102736 (250h9m2.226213986s) [upload=240474]: Waiting for error group managing segments... Apr 18 02:25:51.779193064 (250h9m2.373304314s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.166785373 (250h9m2.760896615s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.184676686 (250h9m2.778787936s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.207075381 (250h9m2.801186622s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.299059914 (250h9m2.893171157s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.302165775 (250h9m2.89627702s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.465614425 (250h9m3.059725667s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.479819483 (250h9m3.073930722s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.533279958 (250h9m3.127391199s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.562759587 (250h9m3.156870833s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.742002496 (250h9m3.336113738s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:53.461038287 (250h9m4.055149536s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1UMjJdYFqUjDcAEamv69RrmqtBPT2wiAk585jBsrv6yxFESjNy, address:145.79.227.11:4743): protocol: expected piece hash; hashstore: write /app/config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/b8/log-00000000000227b8-00005052: no space left on device; EOF Apr 18 02:26:50.666039093 (250h10m1.260150343s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:145.79.227.33:4885): protocol: rpc: dial tcp 145.79.227.33:4885: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666101955 (250h10m1.260213203s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:15jMrmLPcRL8VeYUPzYNPhc2N7qZt4QgTsrSHoo4Zrt4cGrHHK, address:145.79.227.44:4220): protocol: rpc: dial tcp 145.79.227.44:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666324349 (250h10m1.260435599s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12bGaxiZQ1uHmbYMfJ273PHh1jni4AVtTjcBEM38ZFYm2j1rV5g, address:167.17.72.69:4734): protocol: write tcp 139.59.144.22:55810->167.17.72.69:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666367389 (250h10m1.260478826s) [seg_pos={0 0}, upload=240474]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Apr 18 02:26:50.767707041 (250h10m1.361818273s) [upload=240474]: Done uploading segments... Apr 18 02:26:50.767750393 (250h10m1.361861649s) [upload=240474]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 4.186464578s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 36 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX.......X] duration: 1m0.103521763s trace: https://jaeger.dev.storj.io/trace/7a32556c3299f77d FAIL (2026-04-18 02:26:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== pload=240474]: Done begin upload segment. Apr 18 02:25:51.051303397 (250h9m1.645414643s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052559793 (250h9m1.646671043s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052600155 (250h9m1.646711385s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.052639728 (250h9m1.646750954s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053090817 (250h9m1.647202066s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053231933 (250h9m1.647343178s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053514417 (250h9m1.647625667s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053720653 (250h9m1.647831902s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.053903088 (250h9m1.648014338s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054285730 (250h9m1.648396983s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054397651 (250h9m1.648508896s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054802296 (250h9m1.648913546s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.054824840 (250h9m1.648936086s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055258766 (250h9m1.649370033s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055330406 (250h9m1.649441657s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055712191 (250h9m1.649823437s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.055870389 (250h9m1.649981634s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.056357114 (250h9m1.650468364s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Uploading piece... Apr 18 02:25:51.142743972 (250h9m1.736855217s) [noise=true , address=145.79.227.38:4998 , piece_id=OAQUJLFO36DXMTQTPTHQGENTNSH322K5AR6OUNCVPP7BNLG5L4SA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1LV3A9EyGYTRvni2ENn6yYkBdxQiJFW6zLC9vikrezvz6F9dEX, address:145.79.227.38:4998): protocol: rpc: dial tcp 145.79.227.38:4998: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144049380 (250h9m1.738160628s) [noise=true , address=145.79.227.31:4818 , piece_id=COD6Y2NRXL4C4K33H6SXOWAKOOUFHEZWWGAXYTOCZCZEJDQDFROQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:122m1dk7cCwZfUKeq4BChF1UjXSt7swvt1L3HdiSybavXhLEcsb, address:145.79.227.31:4818): protocol: rpc: dial tcp 145.79.227.31:4818: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.144916153 (250h9m1.739027412s) [noise=true , address=145.79.227.49:4355 , piece_id=L7DYM5TPFNKMXFQQD5EEWHOEUPZZA32LG7MM4YVENHZDI3JM62WQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12eA7ZLTkgGyCRsnQSEC8oYM3y9oeB5MQ4FUueJNMb4FLZueNv2, address:145.79.227.49:4355): protocol: rpc: dial tcp 145.79.227.49:4355: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:25:51.632081968 (250h9m2.226193222s) [upload=240474]: Next returned nil segment Apr 18 02:25:51.632102736 (250h9m2.226213986s) [upload=240474]: Waiting for error group managing segments... Apr 18 02:25:51.779193064 (250h9m2.373304314s) [noise=true , address=145.79.227.13:4818 , piece_id=HKLCFYE35YBNVHQRCJD3HKP62TYO3A75I7E4MT2DZLPI4UFCY2DQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.166785373 (250h9m2.760896615s) [noise=true , address=167.17.72.70:4752 , piece_id=2FRYMES3YSXNKWCXX7RIHKHOJITURTPV36JSHMFLBEDMG6ECLZKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.184676686 (250h9m2.778787936s) [noise=true , address=167.17.72.68:4643 , piece_id=SPR27ZSBXU4ZYUQSH5NDUXLHVWRYEDXN5OEXWCLVCNX62VKSSRPA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.207075381 (250h9m2.801186622s) [noise=true , address=167.17.72.71:4791 , piece_id=FE65FAXNTOQVDNZBK7IVRUVFHTTHC7PLLEKRPEGI5S3QEGVWCSHQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.299059914 (250h9m2.893171157s) [noise=true , address=193.149.164.71:4290 , piece_id=V37K6CSZYGYH2ME5ZZ4B2HG6MLUCLZGZD24XKHMTYRM36QKZSG7A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.302165775 (250h9m2.89627702s) [noise=true , address=193.149.164.66:4988 , piece_id=Q6SQLDYRR3UFVRHPPXENV6QYRT6DE7YRHWNFODJRYKSUQAQKA77A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.465614425 (250h9m3.059725667s) [noise=true , address=79.127.205.234:4202 , piece_id=RAQ6ZJQGAX54WPBH4L2YCKVPOGNJIIVGC4Y4DMUSEGLVVIMM72RA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.479819483 (250h9m3.073930722s) [noise=true , address=79.127.205.233:4175 , piece_id=YI6X3CSWPNJGRCDMN4IRVSKBHBY5ATPGRDKKARVTR6DEOAJT5AQA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.533279958 (250h9m3.127391199s) [noise=true , address=79.127.205.237:4228 , piece_id=7Y5TZ4EGLASY4Q3RULNCXJZXCRLACSB4LHLW3NQIBMTPXNQI7E6Q, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.562759587 (250h9m3.156870833s) [noise=true , address=79.127.205.235:4257 , piece_id=L3UYPZLCQB3M2XX5UDQCB4BO7FBHFUJPRH6HCVXSORMUMHARLLKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:52.742002496 (250h9m3.336113738s) [noise=true , address=145.79.227.29:4961 , piece_id=5PLSCMIMF47SC5FTO56TFL7DTX5RYSWIXDHB4CWPXGVYBVML4MRQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: <nil> Apr 18 02:25:53.461038287 (250h9m4.055149536s) [noise=true , address=145.79.227.11:4743 , piece_id=2E66S2TDFWJQ5I77W3O4O7QKBAXFERC4CK3WYJHWWSJEKFXOXN4A, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:1UMjJdYFqUjDcAEamv69RrmqtBPT2wiAk585jBsrv6yxFESjNy, address:145.79.227.11:4743): protocol: expected piece hash; hashstore: write /app/config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/b8/log-00000000000227b8-00005052: no space left on device; EOF Apr 18 02:26:50.666039093 (250h10m1.260150343s) [noise=true , address=145.79.227.33:4885 , piece_id=RJFDD7ZDNMX3QC26DQ2P4ET5ZXQVWWZCWJ2RQATYZ4UJGH3LYXKA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:145.79.227.33:4885): protocol: rpc: dial tcp 145.79.227.33:4885: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666101955 (250h10m1.260213203s) [noise=true , address=145.79.227.44:4220 , piece_id=7HSLW3EM6F75GFT62K5ZEV4R2RXCNOCDJRFLMW4JRY5EVUPWL2KQ, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:15jMrmLPcRL8VeYUPzYNPhc2N7qZt4QgTsrSHoo4Zrt4cGrHHK, address:145.79.227.44:4220): protocol: rpc: dial tcp 145.79.227.44:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666324349 (250h10m1.260435599s) [noise=true , address=167.17.72.69:4734 , piece_id=ZR2CLBAPDR3WNVD7CYOJTN7J4LBG7HQX4CAGHK5LL2RYA7WKGMCA, seg_pos={0 0}, upload=240474]: Done uploading piece. err: ecclient: upload failed (node:12bGaxiZQ1uHmbYMfJ273PHh1jni4AVtTjcBEM38ZFYm2j1rV5g, address:167.17.72.69:4734): protocol: write tcp 139.59.144.22:55810->167.17.72.69:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:26:50.666367389 (250h10m1.260478826s) [seg_pos={0 0}, upload=240474]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Apr 18 02:26:50.767707041 (250h10m1.361818273s) [upload=240474]: Done uploading segments... Apr 18 02:26:50.767750393 (250h10m1.361861649s) [upload=240474]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m0.103546773s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 25 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXX.X.XX..XX.X.....XX..X.......................] duration: 2.268692935s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:26:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.268726503s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [XX....X..........X..........................................] duration: 2.965158453s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:26:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.96518776s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 26 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXX.X.XX..XX.X.....XX..X......................] duration: 1.68815667s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:25:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.688193337s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [XXX....X..........X.........................................] duration: 2.989530797s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:25:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.989562785s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 36 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX.......] duration: 8.722867879s trace: https://jaeger.dev.storj.io/trace/78e9ec05d97d6338 FAIL (2026-04-18 02:25:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 218041s) [upload=240394]: Uploading segments... Apr 18 02:20:32.668377733 (250h3m43.262488977s) [upload=240394]: Got next segment. Inline: false Apr 18 02:20:32.668531312 (250h3m43.262642556s) [upload=240394]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 18 02:20:32.668538982 (250h3m43.262650231s) [upload=240394]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 18 02:20:32.798501929 (250h3m43.392613172s) [seg_pos={0 0}, upload=240394]: Begin upload segment... Apr 18 02:20:32.798619280 (250h3m43.392730526s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.799770801 (250h3m43.393882048s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.801315980 (250h3m43.395427225s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802080311 (250h3m43.39619156s) [seg_pos={0 0}, upload=240394]: Done begin upload segment. Apr 18 02:20:32.802155107 (250h3m43.396266358s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802798648 (250h3m43.396909893s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.805202083 (250h3m43.399313337s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806022799 (250h3m43.40013405s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806689954 (250h3m43.400801208s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807333559 (250h3m43.401444809s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807960573 (250h3m43.402071824s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.808629541 (250h3m43.402740792s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.811002620 (250h3m43.405113872s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.813996142 (250h3m43.408107393s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.815672703 (250h3m43.409783956s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817126477 (250h3m43.411237731s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817333127 (250h3m43.411444377s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.819369687 (250h3m43.413480988s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.820113899 (250h3m43.41422515s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.901705476 (250h3m43.495816721s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12qizevYxbpyxfYKv5wXfWJwdHKXMLbPXfgDWxihHMjCGdpKPqc, address:145.79.227.33:4898): protocol: rpc: dial tcp 145.79.227.33:4898: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.910678821 (250h3m43.504790073s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK, address:145.79.227.49:4367): protocol: rpc: dial tcp 145.79.227.49:4367: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.961158335 (250h3m43.555269584s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:16spDGswJh7AS4m7xYwkJF7p8hARtJdR2hmYw1aWz9NrZDM1jj, address:167.17.72.70:4750): protocol: write tcp 139.59.144.22:50904->167.17.72.70:4750: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.968691165 (250h3m43.562802431s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:1X1An1SwLyzf72pHcN7pYsuVchBWbVWJvbNSm8v3cFcgwzS3P1, address:167.17.72.68:4640): protocol: write tcp 139.59.144.22:59100->167.17.72.68:4640: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:33.366076601 (250h3m43.960187861s) [upload=240394]: Next returned nil segment Apr 18 02:20:33.366100451 (250h3m43.960211708s) [upload=240394]: Waiting for error group managing segments... Apr 18 02:20:33.545413199 (250h3m44.13952444s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.557342012 (250h3m44.151453262s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.966221849 (250h3m44.560333098s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.028680484 (250h3m44.622791735s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.087094362 (250h3m44.681205588s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.096110354 (250h3m44.690221598s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.142002427 (250h3m44.736113668s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.206517179 (250h3m44.800628423s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.215638225 (250h3m44.809749466s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.605492108 (250h3m45.199603338s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.822934405 (250h3m45.417045669s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:36.842912495 (250h3m47.437023763s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:43.279384547 (250h3m53.873495795s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:21:32.358582612 (250h4m42.95269386s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12KuoSWvJCc3mfowPPSJkdCDewkrHZnG6gz1fWcmJRGR5wvHzLL, address:145.79.227.44:4210): protocol: context deadline exceeded; context deadline exceeded Apr 18 02:21:32.358646595 (250h4m42.952757827s) [seg_pos={0 0}, upload=240394]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 18 02:21:32.464561894 (250h4m43.058673133s) [upload=240394]: Done uploading segments... Apr 18 02:21:32.464598829 (250h4m43.05871007s) [upload=240394]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 8.722902412s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 27 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXX.X.XX..XX.X.....XX..X.....................] duration: 1.668006534s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:25:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.668034089s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 37 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX......] duration: 6.67269026s trace: https://jaeger.dev.storj.io/trace/78e9ec05d97d6338 FAIL (2026-04-18 02:24:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 218041s) [upload=240394]: Uploading segments... Apr 18 02:20:32.668377733 (250h3m43.262488977s) [upload=240394]: Got next segment. Inline: false Apr 18 02:20:32.668531312 (250h3m43.262642556s) [upload=240394]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 18 02:20:32.668538982 (250h3m43.262650231s) [upload=240394]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 18 02:20:32.798501929 (250h3m43.392613172s) [seg_pos={0 0}, upload=240394]: Begin upload segment... Apr 18 02:20:32.798619280 (250h3m43.392730526s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.799770801 (250h3m43.393882048s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.801315980 (250h3m43.395427225s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802080311 (250h3m43.39619156s) [seg_pos={0 0}, upload=240394]: Done begin upload segment. Apr 18 02:20:32.802155107 (250h3m43.396266358s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802798648 (250h3m43.396909893s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.805202083 (250h3m43.399313337s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806022799 (250h3m43.40013405s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806689954 (250h3m43.400801208s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807333559 (250h3m43.401444809s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807960573 (250h3m43.402071824s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.808629541 (250h3m43.402740792s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.811002620 (250h3m43.405113872s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.813996142 (250h3m43.408107393s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.815672703 (250h3m43.409783956s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817126477 (250h3m43.411237731s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817333127 (250h3m43.411444377s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.819369687 (250h3m43.413480988s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.820113899 (250h3m43.41422515s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.901705476 (250h3m43.495816721s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12qizevYxbpyxfYKv5wXfWJwdHKXMLbPXfgDWxihHMjCGdpKPqc, address:145.79.227.33:4898): protocol: rpc: dial tcp 145.79.227.33:4898: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.910678821 (250h3m43.504790073s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK, address:145.79.227.49:4367): protocol: rpc: dial tcp 145.79.227.49:4367: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.961158335 (250h3m43.555269584s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:16spDGswJh7AS4m7xYwkJF7p8hARtJdR2hmYw1aWz9NrZDM1jj, address:167.17.72.70:4750): protocol: write tcp 139.59.144.22:50904->167.17.72.70:4750: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.968691165 (250h3m43.562802431s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:1X1An1SwLyzf72pHcN7pYsuVchBWbVWJvbNSm8v3cFcgwzS3P1, address:167.17.72.68:4640): protocol: write tcp 139.59.144.22:59100->167.17.72.68:4640: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:33.366076601 (250h3m43.960187861s) [upload=240394]: Next returned nil segment Apr 18 02:20:33.366100451 (250h3m43.960211708s) [upload=240394]: Waiting for error group managing segments... Apr 18 02:20:33.545413199 (250h3m44.13952444s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.557342012 (250h3m44.151453262s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.966221849 (250h3m44.560333098s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.028680484 (250h3m44.622791735s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.087094362 (250h3m44.681205588s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.096110354 (250h3m44.690221598s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.142002427 (250h3m44.736113668s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.206517179 (250h3m44.800628423s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.215638225 (250h3m44.809749466s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.605492108 (250h3m45.199603338s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.822934405 (250h3m45.417045669s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:36.842912495 (250h3m47.437023763s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:43.279384547 (250h3m53.873495795s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:21:32.358582612 (250h4m42.95269386s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12KuoSWvJCc3mfowPPSJkdCDewkrHZnG6gz1fWcmJRGR5wvHzLL, address:145.79.227.44:4210): protocol: context deadline exceeded; context deadline exceeded Apr 18 02:21:32.358646595 (250h4m42.952757827s) [seg_pos={0 0}, upload=240394]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 18 02:21:32.464561894 (250h4m43.058673133s) [upload=240394]: Done uploading segments... Apr 18 02:21:32.464598829 (250h4m43.05871007s) [upload=240394]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.672720317s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [XXXX....X..........X........................................] duration: 2.958647836s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:24:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.958674812s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.XXXX....X..........X.......................................] duration: 2.956766427s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:24:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.956854587s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 28 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXX.X.XX..XX.X.....XX..X....................] duration: 2.135977425s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:24:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.13601004s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 38 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX.....] duration: 5.643074979s trace: https://jaeger.dev.storj.io/trace/78e9ec05d97d6338 FAIL (2026-04-18 02:23:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 218041s) [upload=240394]: Uploading segments... Apr 18 02:20:32.668377733 (250h3m43.262488977s) [upload=240394]: Got next segment. Inline: false Apr 18 02:20:32.668531312 (250h3m43.262642556s) [upload=240394]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 18 02:20:32.668538982 (250h3m43.262650231s) [upload=240394]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 18 02:20:32.798501929 (250h3m43.392613172s) [seg_pos={0 0}, upload=240394]: Begin upload segment... Apr 18 02:20:32.798619280 (250h3m43.392730526s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.799770801 (250h3m43.393882048s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.801315980 (250h3m43.395427225s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802080311 (250h3m43.39619156s) [seg_pos={0 0}, upload=240394]: Done begin upload segment. Apr 18 02:20:32.802155107 (250h3m43.396266358s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802798648 (250h3m43.396909893s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.805202083 (250h3m43.399313337s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806022799 (250h3m43.40013405s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806689954 (250h3m43.400801208s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807333559 (250h3m43.401444809s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807960573 (250h3m43.402071824s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.808629541 (250h3m43.402740792s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.811002620 (250h3m43.405113872s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.813996142 (250h3m43.408107393s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.815672703 (250h3m43.409783956s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817126477 (250h3m43.411237731s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817333127 (250h3m43.411444377s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.819369687 (250h3m43.413480988s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.820113899 (250h3m43.41422515s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.901705476 (250h3m43.495816721s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12qizevYxbpyxfYKv5wXfWJwdHKXMLbPXfgDWxihHMjCGdpKPqc, address:145.79.227.33:4898): protocol: rpc: dial tcp 145.79.227.33:4898: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.910678821 (250h3m43.504790073s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK, address:145.79.227.49:4367): protocol: rpc: dial tcp 145.79.227.49:4367: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.961158335 (250h3m43.555269584s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:16spDGswJh7AS4m7xYwkJF7p8hARtJdR2hmYw1aWz9NrZDM1jj, address:167.17.72.70:4750): protocol: write tcp 139.59.144.22:50904->167.17.72.70:4750: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.968691165 (250h3m43.562802431s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:1X1An1SwLyzf72pHcN7pYsuVchBWbVWJvbNSm8v3cFcgwzS3P1, address:167.17.72.68:4640): protocol: write tcp 139.59.144.22:59100->167.17.72.68:4640: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:33.366076601 (250h3m43.960187861s) [upload=240394]: Next returned nil segment Apr 18 02:20:33.366100451 (250h3m43.960211708s) [upload=240394]: Waiting for error group managing segments... Apr 18 02:20:33.545413199 (250h3m44.13952444s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.557342012 (250h3m44.151453262s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.966221849 (250h3m44.560333098s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.028680484 (250h3m44.622791735s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.087094362 (250h3m44.681205588s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.096110354 (250h3m44.690221598s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.142002427 (250h3m44.736113668s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.206517179 (250h3m44.800628423s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.215638225 (250h3m44.809749466s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.605492108 (250h3m45.199603338s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.822934405 (250h3m45.417045669s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:36.842912495 (250h3m47.437023763s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:43.279384547 (250h3m53.873495795s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:21:32.358582612 (250h4m42.95269386s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12KuoSWvJCc3mfowPPSJkdCDewkrHZnG6gz1fWcmJRGR5wvHzLL, address:145.79.227.44:4210): protocol: context deadline exceeded; context deadline exceeded Apr 18 02:21:32.358646595 (250h4m42.952757827s) [seg_pos={0 0}, upload=240394]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 18 02:21:32.464561894 (250h4m43.058673133s) [upload=240394]: Done uploading segments... Apr 18 02:21:32.464598829 (250h4m43.05871007s) [upload=240394]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.643093948s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [X.XXXX....X..........X......................................] duration: 3.01127484s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:23:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.01133818s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 29 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXX.X.XX..XX.X.....XX..X...................] duration: 1.842705289s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:23:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.842786728s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 39 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX....] duration: 3.595546859s trace: https://jaeger.dev.storj.io/trace/78e9ec05d97d6338 FAIL (2026-04-18 02:23:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 218041s) [upload=240394]: Uploading segments... Apr 18 02:20:32.668377733 (250h3m43.262488977s) [upload=240394]: Got next segment. Inline: false Apr 18 02:20:32.668531312 (250h3m43.262642556s) [upload=240394]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 18 02:20:32.668538982 (250h3m43.262650231s) [upload=240394]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 18 02:20:32.798501929 (250h3m43.392613172s) [seg_pos={0 0}, upload=240394]: Begin upload segment... Apr 18 02:20:32.798619280 (250h3m43.392730526s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.799770801 (250h3m43.393882048s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.801315980 (250h3m43.395427225s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802080311 (250h3m43.39619156s) [seg_pos={0 0}, upload=240394]: Done begin upload segment. Apr 18 02:20:32.802155107 (250h3m43.396266358s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802798648 (250h3m43.396909893s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.805202083 (250h3m43.399313337s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806022799 (250h3m43.40013405s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806689954 (250h3m43.400801208s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807333559 (250h3m43.401444809s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807960573 (250h3m43.402071824s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.808629541 (250h3m43.402740792s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.811002620 (250h3m43.405113872s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.813996142 (250h3m43.408107393s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.815672703 (250h3m43.409783956s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817126477 (250h3m43.411237731s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817333127 (250h3m43.411444377s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.819369687 (250h3m43.413480988s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.820113899 (250h3m43.41422515s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.901705476 (250h3m43.495816721s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12qizevYxbpyxfYKv5wXfWJwdHKXMLbPXfgDWxihHMjCGdpKPqc, address:145.79.227.33:4898): protocol: rpc: dial tcp 145.79.227.33:4898: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.910678821 (250h3m43.504790073s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK, address:145.79.227.49:4367): protocol: rpc: dial tcp 145.79.227.49:4367: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.961158335 (250h3m43.555269584s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:16spDGswJh7AS4m7xYwkJF7p8hARtJdR2hmYw1aWz9NrZDM1jj, address:167.17.72.70:4750): protocol: write tcp 139.59.144.22:50904->167.17.72.70:4750: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.968691165 (250h3m43.562802431s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:1X1An1SwLyzf72pHcN7pYsuVchBWbVWJvbNSm8v3cFcgwzS3P1, address:167.17.72.68:4640): protocol: write tcp 139.59.144.22:59100->167.17.72.68:4640: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:33.366076601 (250h3m43.960187861s) [upload=240394]: Next returned nil segment Apr 18 02:20:33.366100451 (250h3m43.960211708s) [upload=240394]: Waiting for error group managing segments... Apr 18 02:20:33.545413199 (250h3m44.13952444s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.557342012 (250h3m44.151453262s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.966221849 (250h3m44.560333098s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.028680484 (250h3m44.622791735s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.087094362 (250h3m44.681205588s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.096110354 (250h3m44.690221598s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.142002427 (250h3m44.736113668s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.206517179 (250h3m44.800628423s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.215638225 (250h3m44.809749466s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.605492108 (250h3m45.199603338s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.822934405 (250h3m45.417045669s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:36.842912495 (250h3m47.437023763s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:43.279384547 (250h3m53.873495795s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:21:32.358582612 (250h4m42.95269386s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12KuoSWvJCc3mfowPPSJkdCDewkrHZnG6gz1fWcmJRGR5wvHzLL, address:145.79.227.44:4210): protocol: context deadline exceeded; context deadline exceeded Apr 18 02:21:32.358646595 (250h4m42.952757827s) [seg_pos={0 0}, upload=240394]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 18 02:21:32.464561894 (250h4m43.058673133s) [upload=240394]: Done uploading segments... Apr 18 02:21:32.464598829 (250h4m43.05871007s) [upload=240394]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.595584106s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 40 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX...] duration: 3.577720618s trace: https://jaeger.dev.storj.io/trace/78e9ec05d97d6338 FAIL (2026-04-18 02:23:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 218041s) [upload=240394]: Uploading segments... Apr 18 02:20:32.668377733 (250h3m43.262488977s) [upload=240394]: Got next segment. Inline: false Apr 18 02:20:32.668531312 (250h3m43.262642556s) [upload=240394]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 18 02:20:32.668538982 (250h3m43.262650231s) [upload=240394]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 18 02:20:32.798501929 (250h3m43.392613172s) [seg_pos={0 0}, upload=240394]: Begin upload segment... Apr 18 02:20:32.798619280 (250h3m43.392730526s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.799770801 (250h3m43.393882048s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.801315980 (250h3m43.395427225s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802080311 (250h3m43.39619156s) [seg_pos={0 0}, upload=240394]: Done begin upload segment. Apr 18 02:20:32.802155107 (250h3m43.396266358s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802798648 (250h3m43.396909893s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.805202083 (250h3m43.399313337s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806022799 (250h3m43.40013405s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806689954 (250h3m43.400801208s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807333559 (250h3m43.401444809s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807960573 (250h3m43.402071824s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.808629541 (250h3m43.402740792s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.811002620 (250h3m43.405113872s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.813996142 (250h3m43.408107393s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.815672703 (250h3m43.409783956s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817126477 (250h3m43.411237731s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817333127 (250h3m43.411444377s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.819369687 (250h3m43.413480988s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.820113899 (250h3m43.41422515s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.901705476 (250h3m43.495816721s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12qizevYxbpyxfYKv5wXfWJwdHKXMLbPXfgDWxihHMjCGdpKPqc, address:145.79.227.33:4898): protocol: rpc: dial tcp 145.79.227.33:4898: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.910678821 (250h3m43.504790073s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK, address:145.79.227.49:4367): protocol: rpc: dial tcp 145.79.227.49:4367: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.961158335 (250h3m43.555269584s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:16spDGswJh7AS4m7xYwkJF7p8hARtJdR2hmYw1aWz9NrZDM1jj, address:167.17.72.70:4750): protocol: write tcp 139.59.144.22:50904->167.17.72.70:4750: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.968691165 (250h3m43.562802431s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:1X1An1SwLyzf72pHcN7pYsuVchBWbVWJvbNSm8v3cFcgwzS3P1, address:167.17.72.68:4640): protocol: write tcp 139.59.144.22:59100->167.17.72.68:4640: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:33.366076601 (250h3m43.960187861s) [upload=240394]: Next returned nil segment Apr 18 02:20:33.366100451 (250h3m43.960211708s) [upload=240394]: Waiting for error group managing segments... Apr 18 02:20:33.545413199 (250h3m44.13952444s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.557342012 (250h3m44.151453262s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.966221849 (250h3m44.560333098s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.028680484 (250h3m44.622791735s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.087094362 (250h3m44.681205588s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.096110354 (250h3m44.690221598s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.142002427 (250h3m44.736113668s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.206517179 (250h3m44.800628423s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.215638225 (250h3m44.809749466s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.605492108 (250h3m45.199603338s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.822934405 (250h3m45.417045669s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:36.842912495 (250h3m47.437023763s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:43.279384547 (250h3m53.873495795s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:21:32.358582612 (250h4m42.95269386s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12KuoSWvJCc3mfowPPSJkdCDewkrHZnG6gz1fWcmJRGR5wvHzLL, address:145.79.227.44:4210): protocol: context deadline exceeded; context deadline exceeded Apr 18 02:21:32.358646595 (250h4m42.952757827s) [seg_pos={0 0}, upload=240394]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 18 02:21:32.464561894 (250h4m43.058673133s) [upload=240394]: Done uploading segments... Apr 18 02:21:32.464598829 (250h4m43.05871007s) [upload=240394]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.577754911s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.X.XXXX....X..........X.....................................] duration: 2.916931064s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:22:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.916966611s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 30 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXX.X.XX..XX.X.....XX..X..................] duration: 2.222217593s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:22:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.222246836s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 31 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXX.X.XX..XX.X.....XX..X.................] duration: 2.224290891s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:22:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.22432222s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 41 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX..] duration: 7.614180813s trace: https://jaeger.dev.storj.io/trace/78e9ec05d97d6338 FAIL (2026-04-18 02:22:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 218041s) [upload=240394]: Uploading segments... Apr 18 02:20:32.668377733 (250h3m43.262488977s) [upload=240394]: Got next segment. Inline: false Apr 18 02:20:32.668531312 (250h3m43.262642556s) [upload=240394]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 18 02:20:32.668538982 (250h3m43.262650231s) [upload=240394]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 18 02:20:32.798501929 (250h3m43.392613172s) [seg_pos={0 0}, upload=240394]: Begin upload segment... Apr 18 02:20:32.798619280 (250h3m43.392730526s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.799770801 (250h3m43.393882048s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.801315980 (250h3m43.395427225s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802080311 (250h3m43.39619156s) [seg_pos={0 0}, upload=240394]: Done begin upload segment. Apr 18 02:20:32.802155107 (250h3m43.396266358s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802798648 (250h3m43.396909893s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.805202083 (250h3m43.399313337s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806022799 (250h3m43.40013405s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806689954 (250h3m43.400801208s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807333559 (250h3m43.401444809s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807960573 (250h3m43.402071824s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.808629541 (250h3m43.402740792s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.811002620 (250h3m43.405113872s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.813996142 (250h3m43.408107393s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.815672703 (250h3m43.409783956s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817126477 (250h3m43.411237731s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817333127 (250h3m43.411444377s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.819369687 (250h3m43.413480988s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.820113899 (250h3m43.41422515s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.901705476 (250h3m43.495816721s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12qizevYxbpyxfYKv5wXfWJwdHKXMLbPXfgDWxihHMjCGdpKPqc, address:145.79.227.33:4898): protocol: rpc: dial tcp 145.79.227.33:4898: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.910678821 (250h3m43.504790073s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK, address:145.79.227.49:4367): protocol: rpc: dial tcp 145.79.227.49:4367: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.961158335 (250h3m43.555269584s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:16spDGswJh7AS4m7xYwkJF7p8hARtJdR2hmYw1aWz9NrZDM1jj, address:167.17.72.70:4750): protocol: write tcp 139.59.144.22:50904->167.17.72.70:4750: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.968691165 (250h3m43.562802431s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:1X1An1SwLyzf72pHcN7pYsuVchBWbVWJvbNSm8v3cFcgwzS3P1, address:167.17.72.68:4640): protocol: write tcp 139.59.144.22:59100->167.17.72.68:4640: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:33.366076601 (250h3m43.960187861s) [upload=240394]: Next returned nil segment Apr 18 02:20:33.366100451 (250h3m43.960211708s) [upload=240394]: Waiting for error group managing segments... Apr 18 02:20:33.545413199 (250h3m44.13952444s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.557342012 (250h3m44.151453262s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.966221849 (250h3m44.560333098s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.028680484 (250h3m44.622791735s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.087094362 (250h3m44.681205588s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.096110354 (250h3m44.690221598s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.142002427 (250h3m44.736113668s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.206517179 (250h3m44.800628423s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.215638225 (250h3m44.809749466s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.605492108 (250h3m45.199603338s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.822934405 (250h3m45.417045669s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:36.842912495 (250h3m47.437023763s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:43.279384547 (250h3m53.873495795s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:21:32.358582612 (250h4m42.95269386s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12KuoSWvJCc3mfowPPSJkdCDewkrHZnG6gz1fWcmJRGR5wvHzLL, address:145.79.227.44:4210): protocol: context deadline exceeded; context deadline exceeded Apr 18 02:21:32.358646595 (250h4m42.952757827s) [seg_pos={0 0}, upload=240394]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 18 02:21:32.464561894 (250h4m43.058673133s) [upload=240394]: Done uploading segments... Apr 18 02:21:32.464598829 (250h4m43.05871007s) [upload=240394]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 7.614223366s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [..X.XXXX....X..........X....................................] duration: 3.074357308s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:21:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.074404934s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 42 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX.] duration: 9.425529272s trace: https://jaeger.dev.storj.io/trace/78e9ec05d97d6338 FAIL (2026-04-18 02:21:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 218041s) [upload=240394]: Uploading segments... Apr 18 02:20:32.668377733 (250h3m43.262488977s) [upload=240394]: Got next segment. Inline: false Apr 18 02:20:32.668531312 (250h3m43.262642556s) [upload=240394]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 18 02:20:32.668538982 (250h3m43.262650231s) [upload=240394]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 18 02:20:32.798501929 (250h3m43.392613172s) [seg_pos={0 0}, upload=240394]: Begin upload segment... Apr 18 02:20:32.798619280 (250h3m43.392730526s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.799770801 (250h3m43.393882048s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.801315980 (250h3m43.395427225s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802080311 (250h3m43.39619156s) [seg_pos={0 0}, upload=240394]: Done begin upload segment. Apr 18 02:20:32.802155107 (250h3m43.396266358s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802798648 (250h3m43.396909893s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.805202083 (250h3m43.399313337s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806022799 (250h3m43.40013405s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806689954 (250h3m43.400801208s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807333559 (250h3m43.401444809s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807960573 (250h3m43.402071824s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.808629541 (250h3m43.402740792s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.811002620 (250h3m43.405113872s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.813996142 (250h3m43.408107393s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.815672703 (250h3m43.409783956s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817126477 (250h3m43.411237731s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817333127 (250h3m43.411444377s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.819369687 (250h3m43.413480988s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.820113899 (250h3m43.41422515s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.901705476 (250h3m43.495816721s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12qizevYxbpyxfYKv5wXfWJwdHKXMLbPXfgDWxihHMjCGdpKPqc, address:145.79.227.33:4898): protocol: rpc: dial tcp 145.79.227.33:4898: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.910678821 (250h3m43.504790073s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK, address:145.79.227.49:4367): protocol: rpc: dial tcp 145.79.227.49:4367: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.961158335 (250h3m43.555269584s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:16spDGswJh7AS4m7xYwkJF7p8hARtJdR2hmYw1aWz9NrZDM1jj, address:167.17.72.70:4750): protocol: write tcp 139.59.144.22:50904->167.17.72.70:4750: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.968691165 (250h3m43.562802431s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:1X1An1SwLyzf72pHcN7pYsuVchBWbVWJvbNSm8v3cFcgwzS3P1, address:167.17.72.68:4640): protocol: write tcp 139.59.144.22:59100->167.17.72.68:4640: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:33.366076601 (250h3m43.960187861s) [upload=240394]: Next returned nil segment Apr 18 02:20:33.366100451 (250h3m43.960211708s) [upload=240394]: Waiting for error group managing segments... Apr 18 02:20:33.545413199 (250h3m44.13952444s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.557342012 (250h3m44.151453262s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.966221849 (250h3m44.560333098s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.028680484 (250h3m44.622791735s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.087094362 (250h3m44.681205588s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.096110354 (250h3m44.690221598s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.142002427 (250h3m44.736113668s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.206517179 (250h3m44.800628423s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.215638225 (250h3m44.809749466s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.605492108 (250h3m45.199603338s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.822934405 (250h3m45.417045669s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:36.842912495 (250h3m47.437023763s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:43.279384547 (250h3m53.873495795s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:21:32.358582612 (250h4m42.95269386s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12KuoSWvJCc3mfowPPSJkdCDewkrHZnG6gz1fWcmJRGR5wvHzLL, address:145.79.227.44:4210): protocol: context deadline exceeded; context deadline exceeded Apr 18 02:21:32.358646595 (250h4m42.952757827s) [seg_pos={0 0}, upload=240394]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 18 02:21:32.464561894 (250h4m43.058673133s) [upload=240394]: Done uploading segments... Apr 18 02:21:32.464598829 (250h4m43.05871007s) [upload=240394]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 9.425554752s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 43 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........XX] duration: 1m0.108412584s trace: https://jaeger.dev.storj.io/trace/78e9ec05d97d6338 FAIL (2026-04-18 02:21:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 218041s) [upload=240394]: Uploading segments... Apr 18 02:20:32.668377733 (250h3m43.262488977s) [upload=240394]: Got next segment. Inline: false Apr 18 02:20:32.668531312 (250h3m43.262642556s) [upload=240394]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 18 02:20:32.668538982 (250h3m43.262650231s) [upload=240394]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 18 02:20:32.798501929 (250h3m43.392613172s) [seg_pos={0 0}, upload=240394]: Begin upload segment... Apr 18 02:20:32.798619280 (250h3m43.392730526s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.799770801 (250h3m43.393882048s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.801315980 (250h3m43.395427225s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802080311 (250h3m43.39619156s) [seg_pos={0 0}, upload=240394]: Done begin upload segment. Apr 18 02:20:32.802155107 (250h3m43.396266358s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.802798648 (250h3m43.396909893s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.805202083 (250h3m43.399313337s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806022799 (250h3m43.40013405s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.806689954 (250h3m43.400801208s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807333559 (250h3m43.401444809s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.807960573 (250h3m43.402071824s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.808629541 (250h3m43.402740792s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.811002620 (250h3m43.405113872s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.813996142 (250h3m43.408107393s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.815672703 (250h3m43.409783956s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817126477 (250h3m43.411237731s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.817333127 (250h3m43.411444377s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.819369687 (250h3m43.413480988s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.820113899 (250h3m43.41422515s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Uploading piece... Apr 18 02:20:32.901705476 (250h3m43.495816721s) [noise=true , address=145.79.227.33:4898 , piece_id=GZLJXPWCKPUPN2DT4XMKJP7NCOAMLHOZFHKPHXZXWUGSDUR3I5BA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12qizevYxbpyxfYKv5wXfWJwdHKXMLbPXfgDWxihHMjCGdpKPqc, address:145.79.227.33:4898): protocol: rpc: dial tcp 145.79.227.33:4898: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.910678821 (250h3m43.504790073s) [noise=true , address=145.79.227.49:4367 , piece_id=TAJFZHQTZOYH37C4VKAYY6RJOMEOFM3HVHSCIHAWBFSFHHGJYHNA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK, address:145.79.227.49:4367): protocol: rpc: dial tcp 145.79.227.49:4367: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.961158335 (250h3m43.555269584s) [noise=true , address=167.17.72.70:4750 , piece_id=LDY7KHBEAITT2PPIXF3HTMX7P35GAUHTJ3GKVJ4HFEFNAYWAMCEA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:16spDGswJh7AS4m7xYwkJF7p8hARtJdR2hmYw1aWz9NrZDM1jj, address:167.17.72.70:4750): protocol: write tcp 139.59.144.22:50904->167.17.72.70:4750: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.968691165 (250h3m43.562802431s) [noise=true , address=167.17.72.68:4640 , piece_id=XRRX6NCKZOGDH4DRIFQRJVXWJLKKP4EMY3FH7Q6CJK2ET2I5DMJA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:1X1An1SwLyzf72pHcN7pYsuVchBWbVWJvbNSm8v3cFcgwzS3P1, address:167.17.72.68:4640): protocol: write tcp 139.59.144.22:59100->167.17.72.68:4640: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:33.366076601 (250h3m43.960187861s) [upload=240394]: Next returned nil segment Apr 18 02:20:33.366100451 (250h3m43.960211708s) [upload=240394]: Waiting for error group managing segments... Apr 18 02:20:33.545413199 (250h3m44.13952444s) [noise=true , address=145.79.227.21:4480 , piece_id=O6CJTOSH32EXOFM2RMLWOHXQWCWDC7O6MQXDA47AEHV3Z3PR6DKQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.557342012 (250h3m44.151453262s) [noise=true , address=145.79.227.13:4819 , piece_id=YZQI6Y5XCBAG67PJU2RRXOUNHL337OQTVSZQFXQMONB5N2BKPBMQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:33.966221849 (250h3m44.560333098s) [noise=true , address=167.17.72.71:4814 , piece_id=HBFGG3KJLQDUAKTZIGAHKFIQQG57CIW25V2O62SIXNP5UONHVOZQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.028680484 (250h3m44.622791735s) [noise=true , address=145.79.227.51:4424 , piece_id=JWTXMHLA65FORFSUA66LLNMD4FOYVLR7DKYSWAETZLHBWWGV3ZFA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.087094362 (250h3m44.681205588s) [noise=true , address=64.49.13.27:4964 , piece_id=TD6XIY7SC57JZMKI4NIYMPQ7CVI3YVFP7DG5WSMWDV2CS45YZ3UA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.096110354 (250h3m44.690221598s) [noise=true , address=193.149.164.71:4297 , piece_id=TI3KYOJEKKU52LTRJSTDTIL7Q3WOJT37OH7SNAU2SCOKARGVBV6A, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.142002427 (250h3m44.736113668s) [noise=true , address=79.127.205.235:4263 , piece_id=CYYAHCUSLBHF57GUH3ONAMWAVJ6GI42PD4GA4V645U2HGW7QNDUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.206517179 (250h3m44.800628423s) [noise=true , address=79.127.205.234:4177 , piece_id=TBZKC4KF2GNTVIRE6EW4V4BRSCVTWATGF2AUSWOIFV5LIJOMCJUQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.215638225 (250h3m44.809749466s) [noise=true , address=79.127.205.233:4151 , piece_id=QMUWF2A25PTGCD3MSRBYUWNF4VIYE34AZ4J64LJIZQ44PTZDFXVQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.605492108 (250h3m45.199603338s) [noise=true , address=145.79.227.22:4523 , piece_id=UTBMHUN7A6ANYHEDWL7LK2O475TURNNFQE5BP7KOGIX6S2DJVITA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:34.822934405 (250h3m45.417045669s) [noise=true , address=77.90.31.10:4546 , piece_id=YGFACWLAWPV77U64BF66UGZ6BDGI3ZBGAMZ7UWIVPZL75H4BHWAQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:36.842912495 (250h3m47.437023763s) [noise=true , address=77.90.31.17:4777 , piece_id=AMQKC26XVEW6ZU7ZQ66M6JZ3YTMUZSVRFZ6AFXZ5NJJ7ZPW2YELA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:20:43.279384547 (250h3m53.873495795s) [noise=true , address=145.79.227.34:4936 , piece_id=IMAWTM7W7FSUGL7W6A3DMDXGBVSQWRETLIUYVPAZBZZO3PLD5GSQ, seg_pos={0 0}, upload=240394]: Done uploading piece. err: <nil> Apr 18 02:21:32.358582612 (250h4m42.95269386s) [noise=true , address=145.79.227.44:4210 , piece_id=YBG7SBXSTSZ76Z6KKNWA3KVIAZC2SUWR7J23WYXNKW3ZG6IC7YUA, seg_pos={0 0}, upload=240394]: Done uploading piece. err: ecclient: upload failed (node:12KuoSWvJCc3mfowPPSJkdCDewkrHZnG6gz1fWcmJRGR5wvHzLL, address:145.79.227.44:4210): protocol: context deadline exceeded; context deadline exceeded Apr 18 02:21:32.358646595 (250h4m42.952757827s) [seg_pos={0 0}, upload=240394]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 18 02:21:32.464561894 (250h4m43.058673133s) [upload=240394]: Done uploading segments... Apr 18 02:21:32.464598829 (250h4m43.05871007s) [upload=240394]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.108429686s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 32 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXX.X.XX..XX.X.....XX..X................] duration: 2.316209609s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:21:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.316249535s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [X..X.XXXX....X..........X...................................] duration: 2.993604884s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:21:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.993642644s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 33 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXX.X.XX..XX.X.....XX..X...............] duration: 3.071711823s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:20:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.071787575s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 43 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........X] duration: 1m0.106035135s trace: https://jaeger.dev.storj.io/trace/7f5904be69aafae4 FAIL (2026-04-18 02:20:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 02:19:32.491037005 (250h2m43.085148256s) [seg_pos={0 0}, upload=240379]: Begin upload segment... Apr 18 02:19:32.491135430 (250h2m43.085246681s) [seg_pos={0 0}, upload=240379]: Done begin upload segment. Apr 18 02:19:32.491214613 (250h2m43.085325862s) [noise=true , address=145.79.227.34:4919 , piece_id=3QWOMVC3FSL2AEPWAHE722FW3BMQGZREXUOGDO3ZBGHANRMA5SVA, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.492334531 (250h2m43.08644578s) [noise=true , address=79.127.205.233:4141 , piece_id=OTSSONZOPRHXH3SK2UX3M4MSJJOK55NEYRBZPYEQ3CTRTXUEKNGQ, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.492615112 (250h2m43.086726362s) [noise=true , address=193.149.164.71:4302 , piece_id=FJQ6XBNIKTSDIRDFV4XUFLC3MACC5RRFVZMWWPROTU462QJD53AA, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.493126881 (250h2m43.087238227s) [noise=true , address=167.17.72.70:4775 , piece_id=PPB6IWMXNRPJHEOQWX72OO4PGU72VXHNLBZXFWUMLXR76OVDWU6Q, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.493319621 (250h2m43.087430866s) [noise=true , address=167.17.72.68:4649 , piece_id=74XUHU6LOXPXMJ5OJ5R4AXLD44GKGUOUPQA7ZYWWMQ2EYVM2MPVQ, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.493737451 (250h2m43.087848702s) [noise=true , address=167.17.72.71:4785 , piece_id=LG6UYDMNS3A7WPI3E7ZJDYOXPO24J7BBGYLGVWNVELOKNO4DXC6Q, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.493966645 (250h2m43.088077896s) [noise=true , address=193.149.164.143:4694 , piece_id=VPQWR2M224DMPNKZ75WZNE36NCZWEALPSKUQREAEQS537GGXCHHA, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.494256061 (250h2m43.088367311s) [noise=true , address=77.90.31.18:4835 , piece_id=IZIGAZF5N5E56XKQY2IUTDJXVB2LXASMCKS3DI2O76FDEE7HGKIQ, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.494595883 (250h2m43.088707133s) [noise=true , address=79.127.205.235:4271 , piece_id=H5BRZX6Y5SLABWEVMEQECXBWTTZWYGALUM4ZTFKCNVZZUOKPOH2Q, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.494719689 (250h2m43.088830939s) [noise=true , address=145.79.227.13:4825 , piece_id=JCSULHZXDHJZG5Y2RR4ETH2IN44ICYDBN5JSEYLJXOPDEHQZU3ZA, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.495208434 (250h2m43.089319684s) [noise=true , address=79.127.205.237:4239 , piece_id=XHFFMDF54WCK544J53DRC6HRTKU3JTQN377URTYDSS4FV2ABQZYQ, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.495222882 (250h2m43.089334125s) [noise=true , address=145.79.227.31:4831 , piece_id=WBJSH4BGBYMNYLCZ544SVT6TIKMFSRFTNTKOH2F7SLLV72FV2Z2Q, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.495768542 (250h2m43.089879792s) [noise=true , address=145.79.227.32:4841 , piece_id=RYDBIFYST6JT4NJ5JFWSMYSDBQDHSXS7WVEAPSRLV3HMNGFK3P5Q, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.495827321 (250h2m43.089938571s) [noise=true , address=64.49.13.137:4495 , piece_id=OQNQEMBW26XSR3APXU77T2NZL7UNJ5GPSCQ6LTT33SBDQGD7RJNA, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.496216683 (250h2m43.090327933s) [noise=true , address=167.17.72.69:4725 , piece_id=GHQ7FWEAUDFNQCOH725CMAX5RFYL6BZZ76J3W7KVGYF2I5A3PSQA, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.496459373 (250h2m43.090570623s) [noise=true , address=145.79.227.49:4351 , piece_id=XFKNU7GGXPHQQF4HTBIKZEFZPUF7S3CBMY3CIUWI7NP2CMV4AGDQ, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.497008955 (250h2m43.091120205s) [noise=true , address=45.91.116.12:4233 , piece_id=SE6DQTU3VKQZUKVVNGLN5H2U3UPBIMIADO2F66PDXXSEQ26BWX5A, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.497562014 (250h2m43.091673266s) [noise=true , address=145.79.227.11:4747 , piece_id=P4H252CL6DF66QY4L7WERBVC2OYXS3ENBFRKLNWVAPHJXENWWQDA, seg_pos={0 0}, upload=240379]: Uploading piece... Apr 18 02:19:32.584780397 (250h2m43.178891648s) [noise=true , address=145.79.227.32:4841 , piece_id=RYDBIFYST6JT4NJ5JFWSMYSDBQDHSXS7WVEAPSRLV3HMNGFK3P5Q, seg_pos={0 0}, upload=240379]: Done uploading piece. err: ecclient: upload failed (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD, address:145.79.227.32:4841): protocol: rpc: dial tcp 145.79.227.32:4841: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:19:32.641797237 (250h2m43.235908494s) [noise=true , address=167.17.72.69:4725 , piece_id=GHQ7FWEAUDFNQCOH725CMAX5RFYL6BZZ76J3W7KVGYF2I5A3PSQA, seg_pos={0 0}, upload=240379]: Done uploading piece. err: ecclient: upload failed (node:1FgFAvkTus7ri5TcLGjMv2LVMa5e4fYsj898RSyjdn1EGBEs51, address:167.17.72.69:4725): protocol: write tcp 139.59.144.22:42416->167.17.72.69:4725: write: connection refused; write tcp 139.59.144.22:42416->167.17.72.69:4725: write: connection refused; piecestore: piecestore close: manager closed: conn closed: op Apr 18 02:19:32.663413498 (250h2m43.257524742s) [noise=true , address=193.149.164.143:4694 , piece_id=VPQWR2M224DMPNKZ75WZNE36NCZWEALPSKUQREAEQS537GGXCHHA, seg_pos={0 0}, upload=240379]: Done uploading piece. err: ecclient: upload failed (node:12ATtVD3XyFxzEqwQpSF4t1MWygqEa4Y1CkNj1s43eud8UtuSmx, address:193.149.164.143:4694): protocol: write tcp 139.59.144.22:47362->193.149.164.143:4694: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:19:33.061483173 (250h2m43.655594404s) [upload=240379]: Next returned nil segment Apr 18 02:19:33.061501944 (250h2m43.655613198s) [upload=240379]: Waiting for error group managing segments... Apr 18 02:19:33.218810366 (250h2m43.812921607s) [noise=true , address=145.79.227.31:4831 , piece_id=WBJSH4BGBYMNYLCZ544SVT6TIKMFSRFTNTKOH2F7SLLV72FV2Z2Q, seg_pos={0 0}, upload=240379]: Done uploading piece. err: <nil> Apr 18 02:19:33.234977819 (250h2m43.829089064s) [noise=true , address=145.79.227.13:4825 , piece_id=JCSULHZXDHJZG5Y2RR4ETH2IN44ICYDBN5JSEYLJXOPDEHQZU3ZA, seg_pos={0 0}, upload=240379]: Done uploading piece. err: <nil> Apr 18 02:19:33.307914131 (250h2m43.902025362s) [noise=true , address=45.91.116.12:4233 , piece_id=SE6DQTU3VKQZUKVVNGLN5H2U3UPBIMIADO2F66PDXXSEQ26BWX5A, seg_pos={0 0}, upload=240379]: Done uploading piece. err: <nil> Apr 18 02:19:33.657449499 (250h2m44.25156074s) [noise=true , address=167.17.72.68:4649 , piece_id=74XUHU6LOXPXMJ5OJ5R4AXLD44GKGUOUPQA7ZYWWMQ2EYVM2MPVQ, seg_pos={0 0}, upload=240379]: Done uploading piece. err: <nil> Apr 18 02:19:33.750748421 (250h2m44.344859667s) [noise=true , address=193.149.164.71:4302 , piece_id=FJQ6XBNIKTSDIRDFV4XUFLC3MACC5RRFVZMWWPROTU462QJD53AA, seg_pos={0 0}, upload=240379]: Done uploading piece. err: <nil> Apr 18 02:19:33.785879656 (250h2m44.379990908s) [noise=true , address=64.49.13.137:4495 , piece_id=OQNQEMBW26XSR3APXU77T2NZL7UNJ5GPSCQ6LTT33SBDQGD7RJNA, seg_pos={0 0}, upload=240379]: Done uploading piece. err: <nil> Apr 18 02:19:33.803816671 (250h2m44.397927921s) [noise=true , address=79.127.205.235:4271 , piece_id=H5BRZX6Y5SLABWEVMEQECXBWTTZWYGALUM4ZTFKCNVZZUOKPOH2Q, seg_pos={0 0}, upload=240379]: Done uploading piece. err: <nil> Apr 18 02:19:33.870857174 (250h2m44.464968417s) [noise=true , address=167.17.72.70:4775 , piece_id=PPB6IWMXNRPJHEOQWX72OO4PGU72VXHNLBZXFWUMLXR76OVDWU6Q, seg_pos={0 0}, upload=240379]: Done uploading piece. err: <nil> Apr 18 02:19:34.043922000 (250h2m44.638033241s) [noise=true , address=79.127.205.233:4141 , piece_id=OTSSONZOPRHXH3SK2UX3M4MSJJOK55NEYRBZPYEQ3CTRTXUEKNGQ, seg_pos={0 0}, upload=240379]: Done uploading piece. err: <nil> Apr 18 02:19:34.069251384 (250h2m44.663362629s) [noise=true , address=79.127.205.237:4239 , piece_id=XHFFMDF54WCK544J53DRC6HRTKU3JTQN377URTYDSS4FV2ABQZYQ, seg_pos={0 0}, upload=240379]: Done uploading piece. err: <nil> Apr 18 02:19:34.966972909 (250h2m45.561084139s) [noise=true , address=77.90.31.18:4835 , piece_id=IZIGAZF5N5E56XKQY2IUTDJXVB2LXASMCKS3DI2O76FDEE7HGKIQ, seg_pos={0 0}, upload=240379]: Done uploading piece. err: <nil> Apr 18 02:19:36.381749411 (250h2m46.975860651s) [noise=true , address=145.79.227.11:4747 , piece_id=P4H252CL6DF66QY4L7WERBVC2OYXS3ENBFRKLNWVAPHJXENWWQDA, seg_pos={0 0}, upload=240379]: Done uploading piece. err: <nil> Apr 18 02:20:32.101199663 (250h3m42.69531098s) [noise=true , address=167.17.72.71:4785 , piece_id=LG6UYDMNS3A7WPI3E7ZJDYOXPO24J7BBGYLGVWNVELOKNO4DXC6Q, seg_pos={0 0}, upload=240379]: Done uploading piece. err: ecclient: upload failed (node:1xtN5HpMjoRrZUuhjjAC8Jj3MEmkm4VFhmkDKtN7iEej9xdXiQ, address:167.17.72.71:4785): protocol: write tcp 139.59.144.22:53314->167.17.72.71:4785: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.101259100 (250h3m42.695370456s) [noise=true , address=145.79.227.49:4351 , piece_id=XFKNU7GGXPHQQF4HTBIKZEFZPUF7S3CBMY3CIUWI7NP2CMV4AGDQ, seg_pos={0 0}, upload=240379]: Done uploading piece. err: ecclient: upload failed (node:16aJgn61bhumVFmXfTv8YUi5JfZedtBqadv8yVcSEjgfgUxuDg, address:145.79.227.49:4351): protocol: rpc: dial tcp 145.79.227.49:4351: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.101641786 (250h3m42.695753036s) [noise=true , address=145.79.227.34:4919 , piece_id=3QWOMVC3FSL2AEPWAHE722FW3BMQGZREXUOGDO3ZBGHANRMA5SVA, seg_pos={0 0}, upload=240379]: Done uploading piece. err: ecclient: upload failed (node:1tViqrJYdX8dCfHMjVU9sDatzUARxbGqvvt1x1C334N5SXoaRu, address:145.79.227.34:4919): protocol: rpc: dial tcp 145.79.227.34:4919: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:20:32.101689778 (250h3m42.69580103s) [seg_pos={0 0}, upload=240379]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Apr 18 02:20:32.204708809 (250h3m42.798820053s) [upload=240379]: Done uploading segments... Apr 18 02:20:32.204805268 (250h3m42.798916516s) [upload=240379]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.106050799s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 9 out of 60 is failed (threshold=3) [XX..X.XXXX....X..........X..................................] duration: 3.742196647s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:20:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.742226054s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 34 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXX.X.XX..XX.X.....XX..X..............] duration: 1.793475402s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:19:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.793506048s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 35 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXX.X.XX..XX.X.....XX..X.............] duration: 1.883333294s trace: https://jaeger.dev.storj.io/trace/15bcaf84944be57a FAIL (2026-04-18 02:19:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.883358235s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [XXX..X.XXXX....X..........X.................................] duration: 3.183235924s trace: https://jaeger.dev.storj.io/trace/9a3abeebd2986c0 FAIL (2026-04-18 02:19:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.183635411s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 43 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXX.XXXXX.XXX.XXX.....X........] duration: 14.068962962s trace: https://jaeger.dev.storj.io/trace/37547cd6b582267 FAIL (2026-04-18 02:18:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 813475s) [upload=240249]: Uploading segments... Apr 18 02:11:34.033162278 (249h54m44.627273526s) [upload=240249]: Got next segment. Inline: false Apr 18 02:11:34.033206060 (249h54m44.627317306s) [upload=240249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 18 02:11:34.033211627 (249h54m44.627322874s) [upload=240249]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 18 02:11:34.143887479 (249h54m44.737998735s) [seg_pos={0 0}, upload=240249]: Begin upload segment... Apr 18 02:11:34.143990724 (249h54m44.738101991s) [seg_pos={0 0}, upload=240249]: Done begin upload segment. Apr 18 02:11:34.144035065 (249h54m44.738146326s) [noise=true , address=45.91.116.11:4130 , piece_id=JQIIGNELLXAFCFPQRIIWURAF7GCENBXN2IX47GDBDCK4663EJ7GA, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.144173984 (249h54m44.738285242s) [noise=true , address=193.149.164.66:4978 , piece_id=PJ5HM5G3FQ3RQOSJPKHNQXZW37T75X3DG7JLXJSM57VL3EAVLCKA, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.145993407 (249h54m44.740104668s) [noise=true , address=167.17.72.71:4798 , piece_id=22HYZVLPXVNAU3PXP7WHCTPIH66AOATUREEYDE67DK74Q7WTPWJA, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.146632130 (249h54m44.740743388s) [noise=true , address=145.79.227.49:4357 , piece_id=WMGQ345LHFZNMTKP5VF22ELQFUSLAUWUXYQ5276VJB4OTDUUPTOQ, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.147131073 (249h54m44.741242331s) [noise=true , address=145.79.227.11:4754 , piece_id=LJ26RH47HLPNGWCTGUHWSU4IOXDBPOYCG5Z53HOJMIW62U7JWQ2Q, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.147645331 (249h54m44.741756591s) [noise=true , address=145.79.227.46:4617 , piece_id=ANMMX6TESJXPONLEV7OS4T2XKS7JKUCL7MW2MZFFJBSILOIETZ6A, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.148170600 (249h54m44.742281878s) [noise=true , address=145.79.227.29:4949 , piece_id=PVG6JK3OTM6CIXK4BLCHQPNL7YGXMNN5WCLPVKLMXDM4L7TSWSNA, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.148718883 (249h54m44.74283014s) [noise=true , address=77.90.31.17:4781 , piece_id=OOITU5Y73A47VRCDTC7ZTOECAJEI4X47SC7EUXKXRORHJZCOGXXQ, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.149236868 (249h54m44.743348126s) [noise=true , address=193.149.164.71:4296 , piece_id=3G2U7UBT3LCXV7BGWOE62GCL445NB2WKXGNRXLEWVCQ3YVGACMEQ, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.149773629 (249h54m44.743884886s) [noise=true , address=167.17.72.69:4721 , piece_id=OBTN55XI73GF7CQAB3EG65TJ7TPZRE7M4ZUHIYURFAF2UWQFQJDA, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.150662409 (249h54m44.744773666s) [noise=true , address=45.91.116.4:4958 , piece_id=DGB5YIEIREDVEGNM4QZCJOULZ3UQKG6Q2EFOABSKB57HWJBET3TA, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.151344081 (249h54m44.745455338s) [noise=true , address=79.127.205.233:4167 , piece_id=UEIOYADM64MO3CDDJ33HCISPVI6T47MZ5ONQHWHZUASQBITAHGKQ, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.152027438 (249h54m44.746138695s) [noise=true , address=64.49.13.169:4706 , piece_id=OEOHJSD3QMU5QLBL6P6HY42IIZJLZEXAHYB5KRGRM5KLE7HX45NQ, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.153002236 (249h54m44.747113497s) [noise=true , address=79.127.205.237:4238 , piece_id=XEL22YVUN5DHKHAMQ225UDALPADXTR363ZLDO2LWA72GAX6OWF2A, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.153235588 (249h54m44.747346846s) [noise=true , address=167.17.72.70:4778 , piece_id=M45E56OYHY6BAXOP37UN2DAVJKDTRICUXRVNQYBMGAMAZVP6EMYQ, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.153808451 (249h54m44.747919708s) [noise=true , address=145.79.227.13:4811 , piece_id=65B6PJJ2RVBDSHNUTMYBKND4JVDOTHHOBTZMSP2FPJIQWZB5E4TA, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.154648585 (249h54m44.748759835s) [noise=true , address=193.149.164.143:4694 , piece_id=ZXLLZD73W4REKK4NWNWWS2CV2H6VCVB6OQDBWPEAPZUZSLUXXNGQ, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.154855923 (249h54m44.748967184s) [noise=true , address=79.127.205.234:4187 , piece_id=4VPAWDXVGPUQJXCJHBDPRV34RC2FVBVX5V4X4POGBPB4C6FH44UA, seg_pos={0 0}, upload=240249]: Uploading piece... Apr 18 02:11:34.243611395 (249h54m44.83772264s) [noise=true , address=145.79.227.29:4949 , piece_id=PVG6JK3OTM6CIXK4BLCHQPNL7YGXMNN5WCLPVKLMXDM4L7TSWSNA, seg_pos={0 0}, upload=240249]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:145.79.227.29:4949): protocol: rpc: dial tcp 145.79.227.29:4949: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:11:34.331213671 (249h54m44.925324921s) [noise=true , address=193.149.164.143:4694 , piece_id=ZXLLZD73W4REKK4NWNWWS2CV2H6VCVB6OQDBWPEAPZUZSLUXXNGQ, seg_pos={0 0}, upload=240249]: Done uploading piece. err: ecclient: upload failed (node:12ATtVD3XyFxzEqwQpSF4t1MWygqEa4Y1CkNj1s43eud8UtuSmx, address:193.149.164.143:4694): protocol: write tcp 139.59.144.22:46688->193.149.164.143:4694: write: connection refused; write tcp 139.59.144.22:46688->193.149.164.143:4694: write: connection refused Apr 18 02:11:34.727604270 (249h54m45.321715519s) [upload=240249]: Next returned nil segment Apr 18 02:11:34.727624301 (249h54m45.32173555s) [upload=240249]: Waiting for error group managing segments... Apr 18 02:11:35.177690531 (249h54m45.77180177s) [noise=true , address=145.79.227.13:4811 , piece_id=65B6PJJ2RVBDSHNUTMYBKND4JVDOTHHOBTZMSP2FPJIQWZB5E4TA, seg_pos={0 0}, upload=240249]: Done uploading piece. err: <nil> Apr 18 02:11:35.180928339 (249h54m45.775039587s) [noise=true , address=45.91.116.11:4130 , piece_id=JQIIGNELLXAFCFPQRIIWURAF7GCENBXN2IX47GDBDCK4663EJ7GA, seg_pos={0 0}, upload=240249]: Done uploading piece. err: <nil> Apr 18 02:11:35.359877754 (249h54m45.953988998s) [noise=true , address=167.17.72.69:4721 , piece_id=OBTN55XI73GF7CQAB3EG65TJ7TPZRE7M4ZUHIYURFAF2UWQFQJDA, seg_pos={0 0}, upload=240249]: Done uploading piece. err: <nil> Apr 18 02:11:35.465790685 (249h54m46.059901925s) [noise=true , address=193.149.164.71:4296 , piece_id=3G2U7UBT3LCXV7BGWOE62GCL445NB2WKXGNRXLEWVCQ3YVGACMEQ, seg_pos={0 0}, upload=240249]: Done uploading piece. err: <nil> Apr 18 02:11:35.490324561 (249h54m46.084435803s) [noise=true , address=79.127.205.237:4238 , piece_id=XEL22YVUN5DHKHAMQ225UDALPADXTR363ZLDO2LWA72GAX6OWF2A, seg_pos={0 0}, upload=240249]: Done uploading piece. err: <nil> Apr 18 02:11:35.599141523 (249h54m46.193252786s) [noise=true , address=79.127.205.234:4187 , piece_id=4VPAWDXVGPUQJXCJHBDPRV34RC2FVBVX5V4X4POGBPB4C6FH44UA, seg_pos={0 0}, upload=240249]: Done uploading piece. err: <nil> Apr 18 02:11:35.746879966 (249h54m46.340991214s) [noise=true , address=193.149.164.66:4978 , piece_id=PJ5HM5G3FQ3RQOSJPKHNQXZW37T75X3DG7JLXJSM57VL3EAVLCKA, seg_pos={0 0}, upload=240249]: Done uploading piece. err: <nil> Apr 18 02:11:35.759854123 (249h54m46.353965364s) [noise=true , address=145.79.227.11:4754 , piece_id=LJ26RH47HLPNGWCTGUHWSU4IOXDBPOYCG5Z53HOJMIW62U7JWQ2Q, seg_pos={0 0}, upload=240249]: Done uploading piece. err: <nil> Apr 18 02:11:36.041654214 (249h54m46.63576546s) [noise=true , address=79.127.205.233:4167 , piece_id=UEIOYADM64MO3CDDJ33HCISPVI6T47MZ5ONQHWHZUASQBITAHGKQ, seg_pos={0 0}, upload=240249]: Done uploading piece. err: <nil> Apr 18 02:11:36.562040461 (249h54m47.156151701s) [noise=true , address=45.91.116.4:4958 , piece_id=DGB5YIEIREDVEGNM4QZCJOULZ3UQKG6Q2EFOABSKB57HWJBET3TA, seg_pos={0 0}, upload=240249]: Done uploading piece. err: <nil> Apr 18 02:11:37.006473786 (249h54m47.600585032s) [noise=true , address=77.90.31.17:4781 , piece_id=OOITU5Y73A47VRCDTC7ZTOECAJEI4X47SC7EUXKXRORHJZCOGXXQ, seg_pos={0 0}, upload=240249]: Done uploading piece. err: <nil> Apr 18 02:11:39.560159622 (249h54m50.154270868s) [noise=true , address=167.17.72.71:4798 , piece_id=22HYZVLPXVNAU3PXP7WHCTPIH66AOATUREEYDE67DK74Q7WTPWJA, seg_pos={0 0}, upload=240249]: Done uploading piece. err: <nil> Apr 18 02:11:40.303280817 (249h54m50.897392063s) [noise=true , address=145.79.227.46:4617 , piece_id=ANMMX6TESJXPONLEV7OS4T2XKS7JKUCL7MW2MZFFJBSILOIETZ6A, seg_pos={0 0}, upload=240249]: Done uploading piece. err: <nil> Apr 18 02:12:33.745029575 (249h55m44.339140825s) [noise=true , address=167.17.72.70:4778 , piece_id=M45E56OYHY6BAXOP37UN2DAVJKDTRICUXRVNQYBMGAMAZVP6EMYQ, seg_pos={0 0}, upload=240249]: Done uploading piece. err: ecclient: upload failed (node:1DWfr9QGBeHeMGUazw9QEvSKJA2Y6q5W8hXLfV8zFCvDTScgwp, address:167.17.72.70:4778): protocol: write tcp 139.59.144.22:33706->167.17.72.70:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 18 02:12:33.745286262 (249h55m44.339397518s) [noise=true , address=145.79.227.49:4357 , piece_id=WMGQ345LHFZNMTKP5VF22ELQFUSLAUWUXYQ5276VJB4OTDUUPTOQ, seg_pos={0 0}, upload=240249]: Done uploading piece. err: ecclient: upload failed (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi, address:145.79.227.49:4357): protocol: context deadline exceeded; context deadline exceeded Apr 18 02:12:33.745517223 (249h55m44.339628472s) [noise=true , address=64.49.13.169:4706 , piece_id=OEOHJSD3QMU5QLBL6P6HY42IIZJLZEXAHYB5KRGRM5KLE7HX45NQ, seg_pos={0 0}, upload=240249]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: expected piece hash; context deadline exceeded; EOF Apr 18 02:12:33.745586366 (249h55m44.339697597s) [seg_pos={0 0}, upload=240249]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 18 02:12:33.851761285 (249h55m44.445872539s) [upload=240249]: Done uploading segments... Apr 18 02:12:33.851916468 (249h55m44.446027742s) [upload=240249]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 14.069002987s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)