Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
2.772570819s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:49:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.772596224s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration:
2.381816971s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:48:57 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.381846749s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
2.718377886s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:48:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.718417768s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................X.....................................] duration:
1.679210293s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:48:20 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.679241342s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................XX....................................] duration:
2.426563871s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:48:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.426654531s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
2.865949262s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:47:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.865976192s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................XX...................................] duration:
2.489110464s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:47:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.489139404s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................X....................................] duration:
1.670433964s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:47:41 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.670465295s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.913430747s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:47:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.91346065s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................X...................................] duration:
2.195385608s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:47:03 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.195898466s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.729170934s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:46:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.729195114s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................XX..................................] duration:
2.412109182s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:46:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.412178927s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................X..................................] duration:
1.549085284s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:46:27 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.549109353s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
2.853844231s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:45:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.853871012s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................X.................................] duration:
2.15203152s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:45:47 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.152072656s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................XX.................................] duration:
2.313535844s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:45:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.313562597s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
2.717822242s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:45:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.717851947s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
2.787762185s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:45:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.787799786s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................XX................................] duration:
2.368027615s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:44:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.368084595s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................X................................] duration:
1.676924932s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:44:59 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.677074002s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............................X...............................] duration:
1.626609133s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:44:20 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.626639116s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........................XX...............................] duration:
2.392031225s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:44:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.39205569s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [............................XX..............................] duration:
2.359509115s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:43:52 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.359533269s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............................X..............................] duration:
1.653843537s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:43:52 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.653861707s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
2.856036488s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:43:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.856070345s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
2.764731854s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:43:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.764759097s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............................X.............................] duration:
1.717269041s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:43:01 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.717309074s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............................XX.............................] duration:
2.263924708s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:42:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.263945431s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............................XX............................] duration:
2.37265701s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:42:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.372683346s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............................X............................] duration:
1.608957691s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:42:18 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.608987811s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................................X...........................] duration:
2.195391466s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:42:03 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.195437294s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
2.743813657s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:41:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.743842168s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............................XX...........................] duration:
2.262769101s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:41:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.262805007s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
2.627136005s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:41:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.627267446s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................X..........................] duration:
1.708044066s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:41:15 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.708070163s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................X.........................] duration:
1.632858056s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:40:40 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.632890913s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [................................XX..........................] duration:
2.395867184s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:40:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.395884308s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
2.728750028s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:40:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.728773348s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................................X........................] duration:
1.671811817s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:39:42 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.671843258s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................................XX.........................] duration:
2.511557636s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:39:21 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.511595104s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
2.775003077s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:39:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.775035882s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................X.......................] duration:
1.621801723s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:39:14 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.621827822s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................................XX........................] duration:
2.399404267s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:38:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.399442752s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.790508943s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:38:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.790535417s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................................XX.......................] duration:
2.390698757s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:38:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.390726994s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.820759468s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:38:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.820883071s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................X......................] duration:
2.32090499s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:38:06 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.320965611s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.788245021s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:38:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.788272055s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................X.....................] duration:
1.68607851s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:37:44 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.686107216s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................................XX......................] duration:
2.366669341s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:37:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.366712837s
core
uplink
us-select-1
|