Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................X...........................................] duration: 1.613834463s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:48:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.61386349s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.331201485s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:48:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.331241077s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................X..........................................] duration: 2.060337826s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:48:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.06042661s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.443208907s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:47:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.443252237s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................X.........................................] duration: 2.095191289s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:47:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.095224297s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.339767692s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:47:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.339799927s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................X........................................] duration: 1.690282341s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:46:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.690311642s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.315765982s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:46:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.315861697s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................X.......................................] duration: 1.632096528s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:45:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.63211811s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.373570967s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:45:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.373602291s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................X......................................] duration: 1.674149894s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:45:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.674179009s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.52043774s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:44:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.520466105s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................X.....................................] duration: 1.537871437s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:43:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.537902343s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................X....................................] duration: 1.719908528s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:43:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.72007353s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.320964405s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:43:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.320991267s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................X...................................] duration: 1.651769414s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:42:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.651799639s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.345413772s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:42:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.345442559s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................X..................................] duration: 2.081797261s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:42:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.081867381s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................X.................................] duration: 2.08094658s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:41:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.080988085s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.348854956s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:41:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.34888032s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................X................................] duration: 1.597678492s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:41:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.597713278s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.423179859s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:41:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.423204709s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................X...............................] duration: 1.658047039s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:40:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.658078627s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.676775353s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:40:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.676814366s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................X..............................] duration: 1.682870564s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:39:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.68289693s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.52707647s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:39:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.52716139s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................X.............................] duration: 1.644724054s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:39:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.644755292s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.322934s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:38:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.322963812s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................X............................] duration: 1.57734276s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:38:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.577374474s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.395867306s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:37:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.39591031s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................X...........................] duration: 2.081633573s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:37:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.081669684s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................X..........................] duration: 1.653954566s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:36:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.653988257s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.301265132s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:36:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.301292108s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................X.........................] duration: 1.678198215s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:36:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.678224354s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.38671053s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:35:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.386815375s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................X........................] duration: 1.693242401s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:35:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.693285998s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.384923972s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:35:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.385038108s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................X.......................] duration: 1.617645961s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:35:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.617752224s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................X......................] duration: 1.623059499s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:34:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.623091404s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.439975528s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:34:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.440002793s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................X.....................] duration: 2.06453635s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:34:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.064566166s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.293816473s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:33:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.29384203s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................X....................] duration: 2.091132205s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:33:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.091165996s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.373780064s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:33:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.373810371s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................X...................] duration: 1.585775981s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:32:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.585882902s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................X..................] duration: 1.58532454s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:32:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.585349195s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.349869628s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:32:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.34990016s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................X.................] duration: 2.122306722s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:31:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.122339913s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.392497535s trace: https://jaeger.dev.storj.io/trace/6fbf8730f87aca76 FAIL (2026-01-28 07:31:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 28 07:17:49.676479742 (303h22m27.560433947s) [upload=266879]: Starting upload Jan 28 07:17:49.676584638 (303h22m27.560538841s) [upload=266879]: Done starting upload Jan 28 07:17:49.676636377 (303h22m27.560590567s) [upload=266879]: Uploading segments... Jan 28 07:17:49.676986869 (303h22m27.560941075s) [upload=266879]: Got next segment. Inline: false Jan 28 07:17:49.677059537 (303h22m27.56101374s) [upload=266879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 28 07:17:49.677067291 (303h22m27.561021495s) [upload=266879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 28 07:17:49.832241395 (303h22m27.716195603s) [seg_pos={0 0}, upload=266879]: Begin upload segment... Jan 28 07:17:49.832346487 (303h22m27.716300689s) [seg_pos={0 0}, upload=266879]: Done begin upload segment. Jan 28 07:17:49.832406382 (303h22m27.716360592s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832419660 (303h22m27.71637386s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832509156 (303h22m27.716463358s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.832990487 (303h22m27.716944694s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833008325 (303h22m27.716962527s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833320262 (303h22m27.717274465s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833521382 (303h22m27.717475581s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833622323 (303h22m27.717576528s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.833977469 (303h22m27.717931668s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834003319 (303h22m27.717957526s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834361445 (303h22m27.718315644s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834473447 (303h22m27.718427655s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.834792088 (303h22m27.718746292s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835272256 (303h22m27.71922646s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.835675434 (303h22m27.719629639s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:49.837043217 (303h22m27.720997418s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Uploading piece... Jan 28 07:17:50.436877102 (303h22m28.320831307s) [upload=266879]: Next returned nil segment Jan 28 07:17:50.436895500 (303h22m28.32084971s) [upload=266879]: Waiting for error group managing segments... Jan 28 07:17:50.567034817 (303h22m28.45098902s) [noise=true , address=77.90.31.18:4805 , piece_id=FMAJD2N7HTG6TMIQE6RFLONGCUBC4SKENCL3C5DJEOCNT56CYBQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.567147706 (303h22m28.451101905s) [noise=true , address=77.90.31.14:4664 , piece_id=3LS2HEBTWG5Z42ULAXN2K6NVXSZHANKVPOTLR4JOJTRKUR3GRR5Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.569653279 (303h22m28.453607499s) [noise=true , address=145.79.227.37:4566 , piece_id=KOT3CODXYOZEOYMZ7LKFLENE3IIKZ5TFKCLKYBSMQCOVG5KICBPQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.572464610 (303h22m28.456418801s) [noise=true , address=145.79.227.10:4403 , piece_id=6IHI2NITQKLG7IWVGYL6IQU7IWO5PWQONJBFAWVZNPDJSFV6M5FA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.575267364 (303h22m28.459221553s) [noise=true , address=145.79.227.50:4373 , piece_id=EJXAG6APVN7PBOZI35YZVH6Q244WIPH6SEPMOUJLIGGEUIQE76GA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576049566 (303h22m28.460003767s) [noise=true , address=145.79.227.15:4876 , piece_id=2WT7T67YVCLHH5LIZRWYZSUGIMIWQYN2SIPOBIQFEJ7LN6WF7QGQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.576708768 (303h22m28.460662967s) [noise=true , address=145.79.227.41:4085 , piece_id=GU5WSV5JNR6TGRS3BN2USORY3QM6I36GTSVGTOM67GLJMIE35UMQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.579136219 (303h22m28.463090409s) [noise=true , address=145.79.227.33:4869 , piece_id=WKQFSBEZ6SALG4TOHHR7BVBALRM5P2S6NFPLTJ2QPOBB4WF2YZQA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.580870099 (303h22m28.46482429s) [noise=true , address=145.79.227.43:4157 , piece_id=F2VIJQGN4DEY5IC42QT3DYMH7ZGAOSIGWLV6W43JMG4FNQH5TV2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.588153532 (303h22m28.472107731s) [noise=true , address=145.79.227.31:4798 , piece_id=ZYYAYR54SAY54TDKV3OP5PYTYIYSOSSKAATXISPNZNYEUV6RACTQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589444099 (303h22m28.473398297s) [noise=true , address=145.79.227.46:4603 , piece_id=ELTPASAB4G5OSJXAFP5CEKBBT25RWDAZL2XM7JJDFQAUFNVACOXQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:17:50.589864421 (303h22m28.47381862s) [noise=true , address=145.79.227.49:4337 , piece_id=CBCVJ6MIWYZVK42FXU47XWF5C6QMTRM6ET7ZLVSMJNUYL2ID4TSQ, seg_pos={0 0}, upload=266879]: Done uploading piece. err: <nil> Jan 28 07:18:19.379453822 (303h22m57.263408037s) [noise=true , address=45.91.116.5:4299 , piece_id=2OXWISYH27DER437MQKSZ4356MEMFRITLWP46LCI3JCUBQ2WBURA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379574546 (303h22m57.263528731s) [noise=true , address=45.91.116.6:4975 , piece_id=BFP2UTI4QF4DW3ZA43UYNI4JBJKDDPLKPR37G6C2ALHZ7DLHDSWA, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:44618->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379702232 (303h22m57.26365644s) [noise=true , address=45.91.116.7:4012 , piece_id=E4REYTDDL6NJFEOQ5F7Z5WFK6TBKNS43YPYMYYLYCADZTK3SUW2A, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: context deadline exceeded; context deadline exceeded Jan 28 07:18:19.379781321 (303h22m57.263735531s) [noise=true , address=45.91.116.14:4190 , piece_id=5UYAYQCR6LNZVVVMMJXNP4HH7JYZDOP6KMAY4LYAI5ANSFSYBQ4Q, seg_pos={0 0}, upload=266879]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:53984->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 28 07:18:19.379814213 (303h22m57.263768416s) [seg_pos={0 0}, upload=266879]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 28 07:18:19.489959618 (303h22m57.373913828s) [upload=266879]: Done uploading segments... Jan 28 07:18:19.490001374 (303h22m57.373955584s) [upload=266879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.392524869s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................X................] duration: 1.570066416s trace: https://jaeger.dev.storj.io/trace/1b35a6c2abc67a0b FAIL (2026-01-28 07:30:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.570092711s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)