Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.38671053s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:35:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.386815375s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................X........................] duration: 1.693242401s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:35:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.693285998s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.384923972s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:35:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.385038108s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................X.......................] duration: 1.617645961s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:35:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.617752224s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................X......................] duration: 1.623059499s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:34:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.623091404s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.439975528s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:34:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.440002793s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................X.....................] duration: 2.06453635s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:34:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.064566166s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.293816473s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:33:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.29384203s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................X....................] duration: 2.091132205s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:33:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.091165996s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.373780064s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:33:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.373810371s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................X...................] duration: 1.585775981s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:32:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.585882902s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................X..................] duration: 1.58532454s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:32:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.585349195s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.349869628s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:32:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.34990016s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................X.................] duration: 2.122306722s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:31:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.122339913s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.392497535s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:31:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.392524869s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................X................] duration: 1.570066416s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:30:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.570092711s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.518476461s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:30:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.518821432s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.326757831s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:29:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.326782918s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................................X...............] duration: 1.627816881s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:28:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.627839312s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.346444119s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:28:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.346475689s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................................X..............] duration: 1.542703218s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:28:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.542732861s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................................X.............] duration: 1.694490819s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:28:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.694519615s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.330316523s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:27:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.330341277s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................................X............] duration: 1.650431866s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:27:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.650457561s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.408900317s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:27:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.408917286s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................................X...........] duration: 1.587586639s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:27:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.587617465s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.341442927s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:26:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.341468372s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................................X..........] duration: 1.562056692s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:26:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.562084812s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 1.516131096s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:26:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.516216238s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.44592619s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:26:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.446102853s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.377926635s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:25:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.377951567s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 1.660831606s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:25:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.660910909s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.359289817s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:24:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.359366264s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 1.602827723s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:24:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.602852683s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 1.608964852s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:24:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.608988939s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.351366885s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:24:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.351396181s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 1.694939066s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:23:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.694976693s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.400940536s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:23:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.400970948s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 1.585049081s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:21:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.585073967s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.422705405s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:21:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.422731166s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.348007084s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:21:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.348052484s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 1.563754513s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:20:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.563782573s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.368686544s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:19:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.368713165s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 1.673756209s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:19:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.673780574s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.4332078s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:19:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.433250035s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 1.692160381s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:19:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.692210487s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.340294374s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:18:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.340431294s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.113393497s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:18:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.113417855s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.000784617s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:18:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000873705s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.190922592s trace: https://jaeger.dev.storj.io/trace/315c8b494df301fc FAIL (2026-01-28 00:23:31 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.190953212s core uplink eu1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)