Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 3.122593839s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:41:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.122625688s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......X....................................................] duration: 1.635065633s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:41:06 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.635098532s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........X...................................................] duration: 1.595010305s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:40:33 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.595037442s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 3.30999317s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:40:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.31002675s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........X..................................................] duration: 1.56619978s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:39:49 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.566229632s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 3.307672468s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:39:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.307717587s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 3.133863795s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:39:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.133898827s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........X.................................................] duration: 2.131089536s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:39:01 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.131117446s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........X................................................] duration: 1.550490303s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:38:25 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.550517434s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 3.242717046s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:38:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.242733007s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............X...............................................] duration: 1.627939938s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:37:47 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.627968188s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 3.428435963s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:36:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.428462445s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............X..............................................] duration: 1.621854266s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:36:52 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.621897103s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............X.............................................] duration: 2.145412384s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:36:05 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.145446864s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 3.076547948s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:36:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.076581s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............X............................................] duration: 2.212595564s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:35:44 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.212618106s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.992528611s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:35:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.992563688s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 3.248531804s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:34:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.248560371s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................X...........................................] duration: 1.679835077s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:34:53 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.679871621s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 3.115859209s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:34:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.115904255s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................X..........................................] duration: 1.761832321s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:34:15 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.761867205s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................X.........................................] duration: 1.555695538s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:33:47 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.555733838s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 3.118640022s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:33:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.118664793s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 3.141759388s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:33:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.141792848s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................X........................................] duration: 2.51225231s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:33:11 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.5122877s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 3.400451762s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:32:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.400473662s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.106399041s trace: https://jaeger.dev.storj.io/trace/174b7068acd9c74d FAIL (2026-03-12 00:32:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 10837 (32h20m37.501980439s) [upload=31037]: Uploading segments... Mar 12 00:31:34.486776050 (32h20m37.502445657s) [upload=31037]: Got next segment. Inline: false Mar 12 00:31:34.486855931 (32h20m37.502525535s) [upload=31037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 12 00:31:34.486862930 (32h20m37.502532538s) [upload=31037]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 12 00:31:34.602695736 (32h20m37.618365349s) [seg_pos={0 0}, upload=31037]: Begin upload segment... Mar 12 00:31:34.602763613 (32h20m37.618433214s) [seg_pos={0 0}, upload=31037]: Done begin upload segment. Mar 12 00:31:34.602822074 (32h20m37.618491684s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.602842924 (32h20m37.618512523s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.603681917 (32h20m37.619351527s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.604424128 (32h20m37.620093734s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605011842 (32h20m37.620681449s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.605528765 (32h20m37.621198373s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606168019 (32h20m37.621837625s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.606859614 (32h20m37.622529223s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607454731 (32h20m37.623124341s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.607930410 (32h20m37.623600012s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.609131954 (32h20m37.624801562s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610330433 (32h20m37.626000044s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.610902935 (32h20m37.626572544s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.612789464 (32h20m37.628459074s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.613671297 (32h20m37.629340914s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.615326520 (32h20m37.630996128s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.618326641 (32h20m37.633996248s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:34.619644361 (32h20m37.635313972s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Uploading piece... Mar 12 00:31:35.179232447 (32h20m38.194902033s) [upload=31037]: Next returned nil segment Mar 12 00:31:35.179255069 (32h20m38.194924673s) [upload=31037]: Waiting for error group managing segments... Mar 12 00:31:35.324345351 (32h20m38.34001496s) [noise=true , address=145.79.227.11:4748 , piece_id=ANRQUH4Q6U76CKX5IJNPB4DT5ZWY2Z6DL7M445CTAMCZY6TPEBIQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.329514593 (32h20m38.345184211s) [noise=true , address=145.79.227.17:4984 , piece_id=6OY4DUI4GNEYOL4YGT2DW7HCZJ5LAA73COBG7BOQ3SDBNJIJN4KA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.332335907 (32h20m38.348005511s) [noise=true , address=45.91.116.10:4101 , piece_id=4CLKQAD47CVND7FENO6QCNB6TSJWDL7IKPVZKB5UC23GQH4V7HAA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.347415595 (32h20m38.363085195s) [noise=true , address=145.79.227.16:4928 , piece_id=374MLV3TYE526GE3SA4H4C5VASLIR5F3CGVR26BL6UYXI2JEX2RA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.683227566 (32h20m38.698897167s) [noise=true , address=167.17.72.69:4724 , piece_id=NICLUZ72ZUDOVJKWCPBDK4ZSSKGZVJ3Y3MAZL2ICCCCOMDM3734Q, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.689845167 (32h20m38.705514768s) [noise=true , address=167.17.72.71:4809 , piece_id=2VMRO3MCIVCWI5DRUONAT7XHVNJWMKP6UCVPFNXF3CQPGP2D5SFQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.698339628 (32h20m38.714009233s) [noise=true , address=167.17.72.68:4657 , piece_id=ZO4UJRKHIFS5IHSQ6PKTZ6AJLLG6BC3DQT5HC3OIO6UPWNP7B43A, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.838364650 (32h20m38.854034259s) [noise=true , address=193.149.164.68:4033 , piece_id=ZHFB4E72IZ4SDFOXKXKIFVBYWZ6L52WSASVEV55LL5GOCXS3I3DA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.859483214 (32h20m38.87515283s) [noise=true , address=64.49.13.138:4482 , piece_id=AS7MWR7GBLRJNHSDGY5OVWCY77IKVW2SVHFTVBZSQUCWRQZMBIZA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.860550874 (32h20m38.876220478s) [noise=true , address=193.149.164.70:4118 , piece_id=34MVMVHDGATXPQIQZ2WVBUJYP5NS35TWOMBU7X5LAUNXRZRXOLHQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.870706156 (32h20m38.886375746s) [noise=true , address=193.149.164.66:4977 , piece_id=VXZEPOAY4KXDW3BUCFHWQT7PAEU5WL2BOYF7KUYRI2V72NQVWAFA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:35.894591310 (32h20m38.910260911s) [noise=true , address=193.149.164.71:4291 , piece_id=EWQTPEA7C6BSMOJJZVFUE3TBF7WF2SHHAKOL3QKO4CPBZPWL35BA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:31:37.536870602 (32h20m40.552540214s) [noise=true , address=79.127.205.237:4236 , piece_id=3BU6NGWK2B4S6H5HXJSMSM3ZJEDZZFMEPTADQXJLOAZ2G5EYHOUQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: <nil> Mar 12 00:32:34.202030668 (32h21m37.217700274s) [noise=true , address=77.90.31.14:4685 , piece_id=DVFFETGMI7KCULN5L7EWGCT7SFHAXPJ7NH2NEDO6O3LBYYCUEOZQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12N4QaMcFs6FM8rCLDLYoqiYJDge5tGMwUZkoJc41YA2NM4RAKG, address:77.90.31.14:4685): protocol: write tcp 139.59.144.22:44908->77.90.31.14:4685: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202352883 (32h21m37.218022493s) [noise=true , address=77.90.31.16:4750 , piece_id=TENSMPQBKXM737MG6RMKQJ2JWZF7UFBEAD7VE27DEJ7KR3ZLXDRQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202537306 (32h21m37.218206917s) [noise=true , address=77.90.31.19:4846 , piece_id=JY6E6P6FTE7H5DLV5F66AILET4BOAQ37677GPJRARJSHXKZBWVCQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: write tcp 139.59.144.22:41948->77.90.31.19:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202626126 (32h21m37.218295733s) [noise=true , address=77.90.31.21:4895 , piece_id=VRQUXW53SGSXBNQRLWZNV3GJWM655MKVWDCV4XUK3GW34CX75PVQ, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.202960912 (32h21m37.218630519s) [noise=true , address=77.90.31.18:4826 , piece_id=U6ZYNCYOHPC4UDLETIDEDOICOT3MAUVNBNK3VXADV2N5Y4PFSXNA, seg_pos={0 0}, upload=31037]: Done uploading piece. err: ecclient: upload failed (node:1v6coC31yHW7RcRYPsoUqoYdgS7mrzUHWGMdH2Hma2QC7qe8JQ, address:77.90.31.18:4826): protocol: rpc: dial tcp 77.90.31.18:4826: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 12 00:32:34.203006347 (32h21m37.218675939s) [seg_pos={0 0}, upload=31037]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 12 00:32:34.306147527 (32h21m37.321817139s) [upload=31037]: Done uploading segments... Mar 12 00:32:34.306191442 (32h21m37.321861053s) [upload=31037]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.106420956s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................X.......................................] duration: 1.619058075s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:32:09 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.619085122s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................X......................................] duration: 1.671899096s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:31:20 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.671927045s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................X.....................................] duration: 1.744444969s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:30:14 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.745310358s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................X....................................] duration: 2.110931031s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:29:09 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.110969031s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................X...................................] duration: 2.141328443s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:28:19 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.141349471s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................X..................................] duration: 2.186050084s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:27:38 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.186081226s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................X.................................] duration: 2.297473116s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:26:31 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.29750195s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................X................................] duration: 1.710987453s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:25:44 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.711571158s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................X...............................] duration: 1.593022888s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:25:23 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.593068496s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................X..............................] duration: 2.061531259s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:25:00 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.061558326s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................X.............................] duration: 1.552381972s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:24:13 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.552409577s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................X............................] duration: 1.603323584s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:23:28 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.603355447s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................X...........................] duration: 2.188130472s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:22:28 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.188161682s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................X..........................] duration: 2.166152539s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:21:16 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.166182292s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................X.........................] duration: 1.681195831s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:20:09 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.681227298s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................X........................] duration: 1.682830336s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:19:21 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.682861245s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................X.......................] duration: 1.681518833s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:18:51 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.681549582s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................X......................] duration: 1.688907642s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:17:43 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.688941972s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................X.....................] duration: 1.60784531s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:16:44 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.607869667s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................X....................] duration: 1.606318012s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:16:03 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.60633524s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................X...................] duration: 1.641100671s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:15:33 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.641128012s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................X..................] duration: 1.688071179s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:14:59 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.688095747s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................X.................] duration: 1.721048771s trace: https://jaeger.dev.storj.io/trace/23bfa65069eba8a5 FAIL (2026-03-12 00:14:02 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.72107211s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)