Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.734703936s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:24:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.734731816s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................................X...] duration:
1.638289088s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:24:43 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.638319396s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................................................XX......] duration:
2.462919234s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:24:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.463105399s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration:
2.381479211s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:24:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.381504972s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.764833159s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:24:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.764859996s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................................................XX....] duration:
2.378996369s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:24:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.379029669s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................................................X..] duration:
1.60671553s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:23:45 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.606746627s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................................................XX...] duration:
2.393420388s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:23:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.393450241s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.715381727s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:23:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.715410526s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................................................XX..] duration:
2.451453191s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:23:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 2.451471955s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................................X.] duration:
1.609904664s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:22:58 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.609927913s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.001643682s trace:
https://jaeger.dev.storj.io/trace/4dcbf2bdf7405449
FAIL (2026-02-08 18:22:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001680065s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................................................XX.] duration:
6.430145508s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:22:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 6.43017152s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
30.113526387s trace:
https://jaeger.dev.storj.io/trace/6d7de065855c854f
FAIL (2026-02-08 18:22:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
oad=509178]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:22:03.960927132 (578h26m41.84488134s) [upload=509178]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:22:04.110748788 (578h26m41.994702997s) [seg_pos={0 0}, upload=509178]: Begin upload segment...
Feb 8 18:22:04.110820336 (578h26m41.994774543s) [seg_pos={0 0}, upload=509178]: Done begin upload segment.
Feb 8 18:22:04.110856501 (578h26m41.994810703s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.115693397 (578h26m41.999647587s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.116499874 (578h26m42.000454086s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.118777725 (578h26m42.002731938s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119400779 (578h26m42.003354998s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.119963979 (578h26m42.003918195s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.120514589 (578h26m42.004468803s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121035040 (578h26m42.004989257s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.121549635 (578h26m42.00550385s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122136143 (578h26m42.006090357s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.122700058 (578h26m42.006654279s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123269841 (578h26m42.007224055s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.123822811 (578h26m42.007777028s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.125362061 (578h26m42.009316274s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.126626714 (578h26m42.010580926s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.128551993 (578h26m42.012506207s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Uploading piece...
Feb 8 18:22:04.749709963 (578h26m42.633664174s) [upload=509178]: Next returned nil segment
Feb 8 18:22:04.749726686 (578h26m42.633680894s) [upload=509178]: Waiting for error group managing segments...
Feb 8 18:22:04.892514286 (578h26m42.776468499s) [noise=true , address=45.91.116.12:4228 , piece_id=J5PHXEAUTYO3L2SCOWJV5DG4HSSCCRDXSU4FRQZPZGUAQNYCUCSA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.892834641 (578h26m42.776788835s) [noise=true , address=45.91.116.9:4262 , piece_id=W4IKJMAXYTG3GMYNSRTWHHJFODGL6O23I74EIBWPHZN2EIYSUGZA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.895637894 (578h26m42.7795921s) [noise=true , address=45.91.116.11:4152 , piece_id=E2JT4VPQLPITKBG7VX3DGLHW3C75DKSXRFSDP4FTSMWJTCE2WNYQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.911053197 (578h26m42.795007406s) [noise=true , address=45.91.116.12:4244 , piece_id=677ZBCVMYCFHEHZXDZLMDNH7CAEH54SYWJTE5AVWFNFH4WGW3TVQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:04.913091544 (578h26m42.797045758s) [noise=true , address=45.91.116.7:4012 , piece_id=TQWEGKVB4YKOQQ5JUPDOUB3NJKXT7SJ3MN2HULE3KSDA6EEKG5KQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:08.025526151 (578h26m45.909480356s) [noise=true , address=79.127.219.38:4818 , piece_id=HBQ65EPXPLTY3RRBDJKOJWJOOEAHV2CDQPKSXJG2JJ4WVWIBGEBA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:10.693673801 (578h26m48.577628002s) [noise=true , address=145.79.227.36:4518 , piece_id=X6IT5WDM5GGJEUTBYRCU7JJT6E6SDR43J7MY25KWSHMENNBDZJOA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:12.731005984 (578h26m50.614960184s) [noise=true , address=79.127.219.46:4894 , piece_id=Y4X54TAT6YVVCN7CBMZBBMTQJHMLPUB2CFOBPP3SEPDAXOE4MIFA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: <nil>
Feb 8 18:22:33.605468726 (578h27m11.489422938s) [noise=true , address=77.90.31.19:4844 , piece_id=YQ4QWJRT22Y7FZ2PSZSYADSBYCIG2RW2J6AGVPJBUWF7RWAQRXGA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: rpc: dial tcp 77.90.31.19:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605738489 (578h27m11.489692708s) [noise=true , address=77.90.31.10:4520 , piece_id=GVEUY3LEGLLZOHGUHZJ342W77346FB5OIJGHYWL6RFWAPN6NJBEQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:58478->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.605844639 (578h27m11.489798909s) [noise=true , address=77.90.31.13:4629 , piece_id=HOYQBJTPUN3ZJT24DYY4KPKNP3WC3PY7L5CUSGNMJZH3LZPVYMQQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60068->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606136012 (578h27m11.490090201s) [noise=true , address=79.127.219.38:4820 , piece_id=DDBMTKBRXR7DCD5YNOWWD4NANY6H2MAARBME5JRMGDC5O55ZTFPA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55526->79.127.219.38:4820: use of closed network connection
Feb 8 18:22:33.606166109 (578h27m11.490120322s) [noise=true , address=77.90.31.21:4880 , piece_id=USK3WG27XI5BWWHW4KD6PKF5QZFF2WC55LLDQ6DYICTR7RL4FEQA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: read tcp 139.59.144.22:60158->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606403948 (578h27m11.490358158s) [noise=true , address=77.90.31.16:4736 , piece_id=I3YMRQLJRC6AGZCS3X24NPS4ZMY6V2RUGMSYVQKA6CQCDP74Y6YA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: read tcp 139.59.144.22:41548->77.90.31.16:4736: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606474964 (578h27m11.490429174s) [noise=true , address=77.90.31.11:4556 , piece_id=BBBH26EPDCSQIHDGRXAK5R6B5P7LS2KINZJ4PX23X5ND4A4MEHIQ, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:43432->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606659904 (578h27m11.490614092s) [noise=true , address=77.90.31.15:4697 , piece_id=BSVS5EKTGONBHXQSSCIIEWSKKHJCZPU57I6HA6JOLVDCK2THAPEA, seg_pos={0 0}, upload=509178]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:49004->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:33.606692396 (578h27m11.490646597s) [seg_pos={0 0}, upload=509178]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:33.717090428 (578h27m11.601044639s) [upload=509178]: Done uploading segments...
Feb 8 18:22:33.717136942 (578h27m11.601091151s) [upload=509178]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 30.113539964s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X] duration:
5.001946099s trace:
https://jaeger.dev.storj.io/trace/42fce211b8a1e546
FAIL (2026-02-08 18:22:07 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.002121201s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.114169942s trace:
https://jaeger.dev.storj.io/trace/7a616219d2fa65dc
FAIL (2026-02-08 18:22:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 8)
======LOGS======
7]: Uploading segments...
Feb 8 18:21:33.653493110 (578h26m11.537447299s) [upload=509167]: Got next segment. Inline: false
Feb 8 18:21:33.653540445 (578h26m11.537494643s) [upload=509167]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 18:21:33.653560945 (578h26m11.537515145s) [upload=509167]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 18:21:33.769706309 (578h26m11.65366052s) [seg_pos={0 0}, upload=509167]: Begin upload segment...
Feb 8 18:21:33.769770443 (578h26m11.653724632s) [seg_pos={0 0}, upload=509167]: Done begin upload segment.
Feb 8 18:21:33.769800973 (578h26m11.653755182s) [noise=true , address=45.91.116.9:4295 , piece_id=RIMBYNXW6JP7ZGG5PE7DNQG6DLKPGU5KFRLUCJNMW4QN2RHWGZLQ, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.769818270 (578h26m11.653772481s) [noise=true , address=77.90.31.13:4629 , piece_id=Q6Y3SU27E4H6HU5FIGZIGUFTIKOULNNMNDUKWQQGMWJ2XJAC7QKQ, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.770659552 (578h26m11.654613759s) [noise=true , address=45.91.116.4:4972 , piece_id=QNXBGGUSNY5HIXO7YRZOCGDJZOO26Q5LBJB5KWBXRDHCLCYCUQDA, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.770885920 (578h26m11.65484013s) [noise=true , address=77.90.31.11:4555 , piece_id=T5HUBJ2HF2Y5YBODEU2ARETDTMCXJQCDXRSPZ465Y246YTTBLVXQ, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.771167593 (578h26m11.655121801s) [noise=true , address=45.91.116.7:4019 , piece_id=AVWHEGSC26B6ZI4O73N4YXME3UBNOWJFKWVCGYOGT6IF3TTKWLAA, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.771599762 (578h26m11.655553971s) [noise=true , address=77.90.31.15:4700 , piece_id=EO4HZCHW55HPZ7R6E7QYZNAFWY4QWH4Z2JHD7MB55SNL5VWQTMTQ, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.771634767 (578h26m11.655588974s) [noise=true , address=77.90.31.16:4736 , piece_id=NSNJGQVUOA6EOYLEDKZHNBX2CLTYQU5HPOM3UTLKZVIHT5YIERAQ, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.772101900 (578h26m11.656056109s) [noise=true , address=79.127.219.38:4818 , piece_id=EKHT7VI4SY3V6477VJEST253ELTLDMMAOJFOPVKPU7HK54G2SYIQ, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.772205767 (578h26m11.656159979s) [noise=true , address=77.90.31.15:4697 , piece_id=JU5VV6BG2HRLDR42O7SOGZRQMBD234MDLLU5UYO6LNOJO6HUCLAA, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.772702331 (578h26m11.65665654s) [noise=true , address=145.79.227.31:4797 , piece_id=DNH2UKTSSINGLTVBVOOP7VO6NKIRWD3IA57R7QFP3X7GQQEQ2NIA, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.772985148 (578h26m11.656939357s) [noise=true , address=45.91.116.9:4262 , piece_id=4C5IYXMHLRI6YLNFH777T2S3Y553RVODUOS6XPE7WZSHVBCEMX6A, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.773179973 (578h26m11.657134182s) [noise=true , address=77.90.31.12:4589 , piece_id=3YXUL2XOLXWZ7WICRM2CFCV4ZLZJ6EFWCYMLC2CZZVRI33HDSHXA, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.773574670 (578h26m11.657528885s) [noise=true , address=45.91.116.8:4079 , piece_id=LVBJYRBXDZ5UNZPKDGD4GK7B3GI7TPH6PCRSODS4XDFJKFI6OFBQ, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.773633998 (578h26m11.657588206s) [noise=true , address=77.90.31.17:4773 , piece_id=JUBS3FXXHKYQ7XQXOCBYCT4CFRXTPW77UNB35TIGWY7AOXTQWSBA, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.774102615 (578h26m11.658056824s) [noise=true , address=77.90.31.19:4842 , piece_id=CRVRRMI6FHYMFBP6IE5KW4D4Y5IU7UNCEFYXL3IJZ24EXU3V7FRQ, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:33.774544862 (578h26m11.658499072s) [noise=true , address=45.91.116.12:4255 , piece_id=DAKFUPFZBKKM6MUO5Z7VMAJMDV4D3NJHIJ5FRWBTKYZ2B2T3MCIQ, seg_pos={0 0}, upload=509167]: Uploading piece...
Feb 8 18:21:34.388548240 (578h26m12.272502451s) [upload=509167]: Next returned nil segment
Feb 8 18:21:34.388572620 (578h26m12.272526832s) [upload=509167]: Waiting for error group managing segments...
Feb 8 18:21:34.520812906 (578h26m12.404767114s) [noise=true , address=45.91.116.12:4255 , piece_id=DAKFUPFZBKKM6MUO5Z7VMAJMDV4D3NJHIJ5FRWBTKYZ2B2T3MCIQ, seg_pos={0 0}, upload=509167]: Done uploading piece. err: <nil>
Feb 8 18:21:34.522239660 (578h26m12.406193858s) [noise=true , address=45.91.116.4:4972 , piece_id=QNXBGGUSNY5HIXO7YRZOCGDJZOO26Q5LBJB5KWBXRDHCLCYCUQDA, seg_pos={0 0}, upload=509167]: Done uploading piece. err: <nil>
Feb 8 18:21:34.523366576 (578h26m12.407320783s) [noise=true , address=45.91.116.9:4295 , piece_id=RIMBYNXW6JP7ZGG5PE7DNQG6DLKPGU5KFRLUCJNMW4QN2RHWGZLQ, seg_pos={0 0}, upload=509167]: Done uploading piece. err: <nil>
Feb 8 18:21:34.530821054 (578h26m12.414775255s) [noise=true , address=45.91.116.8:4079 , piece_id=LVBJYRBXDZ5UNZPKDGD4GK7B3GI7TPH6PCRSODS4XDFJKFI6OFBQ, seg_pos={0 0}, upload=509167]: Done uploading piece. err: <nil>
Feb 8 18:21:34.533200867 (578h26m12.417155056s) [noise=true , address=145.79.227.31:4797 , piece_id=DNH2UKTSSINGLTVBVOOP7VO6NKIRWD3IA57R7QFP3X7GQQEQ2NIA, seg_pos={0 0}, upload=509167]: Done uploading piece. err: <nil>
Feb 8 18:21:34.535731401 (578h26m12.419685601s) [noise=true , address=45.91.116.7:4019 , piece_id=AVWHEGSC26B6ZI4O73N4YXME3UBNOWJFKWVCGYOGT6IF3TTKWLAA, seg_pos={0 0}, upload=509167]: Done uploading piece. err: <nil>
Feb 8 18:21:34.538890255 (578h26m12.422844455s) [noise=true , address=45.91.116.9:4262 , piece_id=4C5IYXMHLRI6YLNFH777T2S3Y553RVODUOS6XPE7WZSHVBCEMX6A, seg_pos={0 0}, upload=509167]: Done uploading piece. err: <nil>
Feb 8 18:21:46.004972695 (578h26m23.888926901s) [noise=true , address=79.127.219.38:4818 , piece_id=EKHT7VI4SY3V6477VJEST253ELTLDMMAOJFOPVKPU7HK54G2SYIQ, seg_pos={0 0}, upload=509167]: Done uploading piece. err: <nil>
Feb 8 18:22:03.353464117 (578h26m41.237418329s) [noise=true , address=77.90.31.12:4589 , piece_id=3YXUL2XOLXWZ7WICRM2CFCV4ZLZJ6EFWCYMLC2CZZVRI33HDSHXA, seg_pos={0 0}, upload=509167]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: rpc: dial tcp 77.90.31.12:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:03.353516271 (578h26m41.237470475s) [noise=true , address=77.90.31.17:4773 , piece_id=JUBS3FXXHKYQ7XQXOCBYCT4CFRXTPW77UNB35TIGWY7AOXTQWSBA, seg_pos={0 0}, upload=509167]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 139.59.144.22:36528->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:03.353749721 (578h26m41.237703924s) [noise=true , address=77.90.31.11:4555 , piece_id=T5HUBJ2HF2Y5YBODEU2ARETDTMCXJQCDXRSPZ465Y246YTTBLVXQ, seg_pos={0 0}, upload=509167]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: read tcp 139.59.144.22:40224->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:03.354036814 (578h26m41.237991033s) [noise=true , address=77.90.31.13:4629 , piece_id=Q6Y3SU27E4H6HU5FIGZIGUFTIKOULNNMNDUKWQQGMWJ2XJAC7QKQ, seg_pos={0 0}, upload=509167]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 139.59.144.22:60346->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:03.354055319 (578h26m41.238009522s) [noise=true , address=77.90.31.19:4842 , piece_id=CRVRRMI6FHYMFBP6IE5KW4D4Y5IU7UNCEFYXL3IJZ24EXU3V7FRQ, seg_pos={0 0}, upload=509167]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:32922->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:03.354178376 (578h26m41.23813257s) [noise=true , address=77.90.31.16:4736 , piece_id=NSNJGQVUOA6EOYLEDKZHNBX2CLTYQU5HPOM3UTLKZVIHT5YIERAQ, seg_pos={0 0}, upload=509167]: Done uploading piece. err: ecclient: upload failed (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk, address:77.90.31.16:4736): protocol: rpc: dial tcp 77.90.31.16:4736: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:03.354473532 (578h26m41.238427736s) [noise=true , address=77.90.31.15:4697 , piece_id=JU5VV6BG2HRLDR42O7SOGZRQMBD234MDLLU5UYO6LNOJO6HUCLAA, seg_pos={0 0}, upload=509167]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 139.59.144.22:39724->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:03.354684535 (578h26m41.238638745s) [noise=true , address=77.90.31.15:4700 , piece_id=EO4HZCHW55HPZ7R6E7QYZNAFWY4QWH4Z2JHD7MB55SNL5VWQTMTQ, seg_pos={0 0}, upload=509167]: Done uploading piece. err: ecclient: upload failed (node:124p2J9pP2rYD3cM73PQNEp6Ykgmpq2xi6aK8wnC6JTiLvHRb7q, address:77.90.31.15:4700): protocol: read tcp 139.59.144.22:57380->77.90.31.15:4700: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 8 18:22:03.354731380 (578h26m41.238685583s) [seg_pos={0 0}, upload=509167]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil>
Feb 8 18:22:03.465252008 (578h26m41.349206219s) [upload=509167]: Done uploading segments...
Feb 8 18:22:03.465297710 (578h26m41.349251907s) [upload=509167]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8)
Duration: 30.114191508s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
1.578071944s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 05:05:08 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.578097965s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.X..........................................................] duration:
1.797283585s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 05:04:33 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.797330842s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..X.........................................................] duration:
1.720277269s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 05:04:12 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.720303415s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...X........................................................] duration:
1.689734867s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 05:03:36 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.689764114s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....X.......................................................] duration:
1.639579303s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 05:02:42 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.639600214s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....X......................................................] duration:
1.702659806s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 05:01:50 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.702699608s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......X.....................................................] duration:
1.647053122s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 05:01:02 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.647081761s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......X....................................................] duration:
2.204211662s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 05:00:07 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.204241535s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........X...................................................] duration:
1.669439828s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:59:20 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.669466298s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........X..................................................] duration:
1.728772096s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:58:55 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.72880404s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........X.................................................] duration:
1.651271812s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:57:47 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.651293405s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........X................................................] duration:
2.160685643s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:57:16 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.160710059s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............X...............................................] duration:
1.635606617s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:56:12 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.635652533s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............X..............................................] duration:
1.656368628s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:55:57 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.656388846s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............X.............................................] duration:
1.647450136s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:55:42 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.64754864s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............X............................................] duration:
1.627643597s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:55:22 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.627670885s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................X...........................................] duration:
1.669762194s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:54:50 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.669787378s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................X..........................................] duration:
2.046088788s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:54:00 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.046120925s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................X.........................................] duration:
1.589327325s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:53:32 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.589430361s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................X........................................] duration:
2.126701663s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:52:44 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.126752541s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................X.......................................] duration:
1.645456907s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:51:52 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.64548584s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................X......................................] duration:
2.116113261s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:50:38 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.1161968s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................X.....................................] duration:
1.646559176s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:50:21 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.646736236s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................X....................................] duration:
1.548215568s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:49:14 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.54823905s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................X...................................] duration:
1.718634162s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:48:09 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.718660248s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................X..................................] duration:
1.840887148s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:47:39 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.84098295s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................X.................................] duration:
1.840014798s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:47:16 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.840107188s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................X................................] duration:
1.62421968s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:46:28 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.624254455s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............................X...............................] duration:
1.676571964s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:45:56 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.676601291s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............................X..............................] duration:
1.662535729s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:45:23 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.662559594s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............................X.............................] duration:
1.581580531s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:44:19 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.581660958s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............................X............................] duration:
2.203742217s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:43:57 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.203772244s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................................X...........................] duration:
1.645068923s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:43:17 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.645094812s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................X..........................] duration:
1.580039037s trace:
https://jaeger.dev.storj.io/trace/283383923ae96920
FAIL (2026-02-08 04:42:17 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.580062972s
core
uplink
us-select-1
us1-placement-20
|