Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
3.063556924s trace:
https://jaeger.dev.storj.io/trace/b68ffcae175905a
FAIL (2026-03-10 18:40:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
05104 (2h26m26.456874693s) [upload=2343]: Got next segment. Inline: false
Mar 10 18:37:23.441245955 (2h26m26.456915545s) [upload=2343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 18:37:23.441251057 (2h26m26.45692066s) [upload=2343]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 18:37:23.557618294 (2h26m26.573287881s) [seg_pos={0 0}, upload=2343]: Begin upload segment...
Mar 10 18:37:23.557695669 (2h26m26.573365278s) [noise=true , address=77.90.31.13:4645 , piece_id=UXFW5W5TPZBMH6IC7AT5GN2ZDMZTNDJLFH7OMNKWN5JXKAT76SFA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.558906475 (2h26m26.574576087s) [seg_pos={0 0}, upload=2343]: Done begin upload segment.
Mar 10 18:37:23.558987696 (2h26m26.57465731s) [noise=true , address=79.127.219.41:4947 , piece_id=NI3ZGQNFOLUIRH3B4QB3FDXFLVNCYKGX24U4HROKCESWMXNKC54A, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.559601912 (2h26m26.575271515s) [noise=true , address=77.90.31.19:4874 , piece_id=DC4C4HMX2TD6FTW5RPS3O55IHEQ6E7OR6VKKREVKVW7FAFAPZ2XA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.560517706 (2h26m26.57618731s) [noise=true , address=45.91.116.4:4943 , piece_id=EOL7JSKHMYZQPZVM3Q5GN2HJC65YKJWG3KTJMICUA4PLSIERLDZQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.561695181 (2h26m26.577364785s) [noise=true , address=77.90.31.14:4673 , piece_id=XY5KYY3SODVHKX2EM2NJT6FRSDGSKNZFSLOKPP3T4VKC423TMQUA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.562028023 (2h26m26.57769763s) [noise=true , address=193.149.164.68:4068 , piece_id=7WA3675HW4UPV7ZTNJEZIJDB7IIHFWLVHTSXOR5UMLQIXNHE7AXA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.562622388 (2h26m26.578292229s) [noise=true , address=45.91.116.6:4999 , piece_id=DJDQ6BIBJESD4OKWTL6JIACEQ5LU4LROC3WNXEF6APEM734BOXZQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.564570119 (2h26m26.580239738s) [noise=true , address=193.149.164.69:4081 , piece_id=6DNISIGMWMIMA6OICHGJUJOR6NNMXSCJI6B4GO6VKT4MCTMKG3HA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.564708737 (2h26m26.58037834s) [noise=true , address=167.17.72.68:4659 , piece_id=4LQBLKHK5RRKVVYFSPSMDWHWG2QPDHZNG2REAVL2PMWO3VZ36NAQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.566432612 (2h26m26.582102222s) [noise=true , address=167.17.72.69:4727 , piece_id=2PMBDX2OJJD33H3GCSQRJAPZMOV5II3WLZOY5YF4HKE7GYBQU6AQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.567575901 (2h26m26.583245509s) [noise=true , address=77.90.31.15:4717 , piece_id=MLYGZT47MTTYGBLNTPA7DPM7NE6EQ6MPZYFW7KVDLFXR4MD2ORPA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.567768288 (2h26m26.583437899s) [noise=true , address=193.149.164.67:4016 , piece_id=CITHWRKZJJGV4X3VG7CPQPWX2FPJDXU4QMSFMVE6ANNKAIICEUMA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.568437376 (2h26m26.584106976s) [noise=true , address=45.91.116.13:4167 , piece_id=3GJVLC4HXSBZ6D3FNOYONOTF6ZN2YUKPI3N4XFX34E7TTT7KE77Q, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.569626647 (2h26m26.585296258s) [noise=true , address=193.149.164.66:4978 , piece_id=UFBW2KYD7LUKXGBAOB4JOTNEKUE77KZSKFFJ3EFDGHPC7QKPAFXQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.570719385 (2h26m26.586388996s) [noise=true , address=45.91.116.10:4093 , piece_id=HUH66QBVZCXFN7BLJRADGU2H42MKXL3WIICJI6LAWQF2J3YXIGRQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.572104408 (2h26m26.587774016s) [noise=true , address=77.90.31.17:4796 , piece_id=FZMZYUGLKY5FPBA3GEF7UVM5IBNO2KRUSEQCJSUVL2PRRN5AWOMQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.573696158 (2h26m26.589365758s) [noise=true , address=77.90.31.12:4614 , piece_id=MLXXO4453CEWAG3G5WZZLAAX7LWUSFVVWQ3FXFKODALSAWRHLEFA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.575002207 (2h26m26.590671812s) [noise=true , address=145.79.227.17:4000 , piece_id=AU4UBLJAXLH7JNNYKC3YB3UIJAAEWD2Q6S77CG3J6G5WJIWPDXAQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:24.121673515 (2h26m27.137343125s) [upload=2343]: Next returned nil segment
Mar 10 18:37:24.121693564 (2h26m27.137363175s) [upload=2343]: Waiting for error group managing segments...
Mar 10 18:37:24.258918473 (2h26m27.274588073s) [noise=true , address=45.91.116.4:4943 , piece_id=EOL7JSKHMYZQPZVM3Q5GN2HJC65YKJWG3KTJMICUA4PLSIERLDZQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.259101868 (2h26m27.274771472s) [noise=true , address=79.127.219.41:4947 , piece_id=NI3ZGQNFOLUIRH3B4QB3FDXFLVNCYKGX24U4HROKCESWMXNKC54A, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.265791082 (2h26m27.281460686s) [noise=true , address=145.79.227.17:4000 , piece_id=AU4UBLJAXLH7JNNYKC3YB3UIJAAEWD2Q6S77CG3J6G5WJIWPDXAQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.277538368 (2h26m27.293207972s) [noise=true , address=45.91.116.13:4167 , piece_id=3GJVLC4HXSBZ6D3FNOYONOTF6ZN2YUKPI3N4XFX34E7TTT7KE77Q, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.283507633 (2h26m27.299177234s) [noise=true , address=45.91.116.10:4093 , piece_id=HUH66QBVZCXFN7BLJRADGU2H42MKXL3WIICJI6LAWQF2J3YXIGRQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.288254595 (2h26m27.303924202s) [noise=true , address=45.91.116.6:4999 , piece_id=DJDQ6BIBJESD4OKWTL6JIACEQ5LU4LROC3WNXEF6APEM734BOXZQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.650345388 (2h26m27.666014988s) [noise=true , address=167.17.72.68:4659 , piece_id=4LQBLKHK5RRKVVYFSPSMDWHWG2QPDHZNG2REAVL2PMWO3VZ36NAQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.658677204 (2h26m27.674346807s) [noise=true , address=167.17.72.69:4727 , piece_id=2PMBDX2OJJD33H3GCSQRJAPZMOV5II3WLZOY5YF4HKE7GYBQU6AQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.804520312 (2h26m27.820189912s) [noise=true , address=193.149.164.69:4081 , piece_id=6DNISIGMWMIMA6OICHGJUJOR6NNMXSCJI6B4GO6VKT4MCTMKG3HA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.805528476 (2h26m27.821198079s) [noise=true , address=193.149.164.68:4068 , piece_id=7WA3675HW4UPV7ZTNJEZIJDB7IIHFWLVHTSXOR5UMLQIXNHE7AXA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.810879435 (2h26m27.826549059s) [noise=true , address=193.149.164.67:4016 , piece_id=CITHWRKZJJGV4X3VG7CPQPWX2FPJDXU4QMSFMVE6ANNKAIICEUMA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.813793209 (2h26m27.829462813s) [noise=true , address=193.149.164.66:4978 , piece_id=UFBW2KYD7LUKXGBAOB4JOTNEKUE77KZSKFFJ3EFDGHPC7QKPAFXQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:38:23.155284138 (2h27m26.170953744s) [noise=true , address=77.90.31.12:4614 , piece_id=MLXXO4453CEWAG3G5WZZLAAX7LWUSFVVWQ3FXFKODALSAWRHLEFA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:12wKLBwTAdUVAQmQGcafdpUcjiDE89jxdptEwXoAvV6WXf2Xbc3, address:77.90.31.12:4614): protocol: rpc: dial tcp 77.90.31.12:4614: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155359187 (2h27m26.171028792s) [noise=true , address=77.90.31.17:4796 , piece_id=FZMZYUGLKY5FPBA3GEF7UVM5IBNO2KRUSEQCJSUVL2PRRN5AWOMQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1KK6y5NBuZovne8n3gMvAyDcZFasZxDzorvPeHe7CefoFs27UF, address:77.90.31.17:4796): protocol: write tcp 139.59.144.22:52670->77.90.31.17:4796: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155422072 (2h27m26.171091682s) [noise=true , address=77.90.31.15:4717 , piece_id=MLYGZT47MTTYGBLNTPA7DPM7NE6EQ6MPZYFW7KVDLFXR4MD2ORPA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:12ULANtxKNaJviAZFsmHjDQWjG4x2wfmjCwbQtHmn9LpnELmkPc, address:77.90.31.15:4717): protocol: rpc: dial tcp 77.90.31.15:4717: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155489065 (2h27m26.171158674s) [noise=true , address=77.90.31.13:4645 , piece_id=UXFW5W5TPZBMH6IC7AT5GN2ZDMZTNDJLFH7OMNKWN5JXKAT76SFA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: rpc: dial tcp 77.90.31.13:4645: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155521077 (2h27m26.171190679s) [noise=true , address=77.90.31.14:4673 , piece_id=XY5KYY3SODVHKX2EM2NJT6FRSDGSKNZFSLOKPP3T4VKC423TMQUA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1xMwYmwF8fukP4nNY3GNLdZZ3a4YqVr35poQf3Trq7AYUNF7sh, address:77.90.31.14:4673): protocol: rpc: dial tcp 77.90.31.14:4673: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155568492 (2h27m26.171238098s) [noise=true , address=77.90.31.19:4874 , piece_id=DC4C4HMX2TD6FTW5RPS3O55IHEQ6E7OR6VKKREVKVW7FAFAPZ2XA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:14J8ruKPB6P9YBLPhHpCrcLJt3WBYwNo3fGBeJ2Pdp6JfWs3W4, address:77.90.31.19:4874): protocol: rpc: dial tcp 77.90.31.19:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155592304 (2h27m26.171261905s) [seg_pos={0 0}, upload=2343]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Mar 10 18:38:23.260013959 (2h27m26.275683549s) [upload=2343]: Done uploading segments...
Mar 10 18:38:23.260057302 (2h27m26.275726903s) [upload=2343]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.063579517s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............................................X.............] duration:
1.698329287s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:40:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.698342311s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.767751797s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:40:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.76777809s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
3.272315794s trace:
https://jaeger.dev.storj.io/trace/b68ffcae175905a
FAIL (2026-03-10 18:39:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
05104 (2h26m26.456874693s) [upload=2343]: Got next segment. Inline: false
Mar 10 18:37:23.441245955 (2h26m26.456915545s) [upload=2343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 18:37:23.441251057 (2h26m26.45692066s) [upload=2343]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 18:37:23.557618294 (2h26m26.573287881s) [seg_pos={0 0}, upload=2343]: Begin upload segment...
Mar 10 18:37:23.557695669 (2h26m26.573365278s) [noise=true , address=77.90.31.13:4645 , piece_id=UXFW5W5TPZBMH6IC7AT5GN2ZDMZTNDJLFH7OMNKWN5JXKAT76SFA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.558906475 (2h26m26.574576087s) [seg_pos={0 0}, upload=2343]: Done begin upload segment.
Mar 10 18:37:23.558987696 (2h26m26.57465731s) [noise=true , address=79.127.219.41:4947 , piece_id=NI3ZGQNFOLUIRH3B4QB3FDXFLVNCYKGX24U4HROKCESWMXNKC54A, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.559601912 (2h26m26.575271515s) [noise=true , address=77.90.31.19:4874 , piece_id=DC4C4HMX2TD6FTW5RPS3O55IHEQ6E7OR6VKKREVKVW7FAFAPZ2XA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.560517706 (2h26m26.57618731s) [noise=true , address=45.91.116.4:4943 , piece_id=EOL7JSKHMYZQPZVM3Q5GN2HJC65YKJWG3KTJMICUA4PLSIERLDZQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.561695181 (2h26m26.577364785s) [noise=true , address=77.90.31.14:4673 , piece_id=XY5KYY3SODVHKX2EM2NJT6FRSDGSKNZFSLOKPP3T4VKC423TMQUA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.562028023 (2h26m26.57769763s) [noise=true , address=193.149.164.68:4068 , piece_id=7WA3675HW4UPV7ZTNJEZIJDB7IIHFWLVHTSXOR5UMLQIXNHE7AXA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.562622388 (2h26m26.578292229s) [noise=true , address=45.91.116.6:4999 , piece_id=DJDQ6BIBJESD4OKWTL6JIACEQ5LU4LROC3WNXEF6APEM734BOXZQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.564570119 (2h26m26.580239738s) [noise=true , address=193.149.164.69:4081 , piece_id=6DNISIGMWMIMA6OICHGJUJOR6NNMXSCJI6B4GO6VKT4MCTMKG3HA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.564708737 (2h26m26.58037834s) [noise=true , address=167.17.72.68:4659 , piece_id=4LQBLKHK5RRKVVYFSPSMDWHWG2QPDHZNG2REAVL2PMWO3VZ36NAQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.566432612 (2h26m26.582102222s) [noise=true , address=167.17.72.69:4727 , piece_id=2PMBDX2OJJD33H3GCSQRJAPZMOV5II3WLZOY5YF4HKE7GYBQU6AQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.567575901 (2h26m26.583245509s) [noise=true , address=77.90.31.15:4717 , piece_id=MLYGZT47MTTYGBLNTPA7DPM7NE6EQ6MPZYFW7KVDLFXR4MD2ORPA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.567768288 (2h26m26.583437899s) [noise=true , address=193.149.164.67:4016 , piece_id=CITHWRKZJJGV4X3VG7CPQPWX2FPJDXU4QMSFMVE6ANNKAIICEUMA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.568437376 (2h26m26.584106976s) [noise=true , address=45.91.116.13:4167 , piece_id=3GJVLC4HXSBZ6D3FNOYONOTF6ZN2YUKPI3N4XFX34E7TTT7KE77Q, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.569626647 (2h26m26.585296258s) [noise=true , address=193.149.164.66:4978 , piece_id=UFBW2KYD7LUKXGBAOB4JOTNEKUE77KZSKFFJ3EFDGHPC7QKPAFXQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.570719385 (2h26m26.586388996s) [noise=true , address=45.91.116.10:4093 , piece_id=HUH66QBVZCXFN7BLJRADGU2H42MKXL3WIICJI6LAWQF2J3YXIGRQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.572104408 (2h26m26.587774016s) [noise=true , address=77.90.31.17:4796 , piece_id=FZMZYUGLKY5FPBA3GEF7UVM5IBNO2KRUSEQCJSUVL2PRRN5AWOMQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.573696158 (2h26m26.589365758s) [noise=true , address=77.90.31.12:4614 , piece_id=MLXXO4453CEWAG3G5WZZLAAX7LWUSFVVWQ3FXFKODALSAWRHLEFA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.575002207 (2h26m26.590671812s) [noise=true , address=145.79.227.17:4000 , piece_id=AU4UBLJAXLH7JNNYKC3YB3UIJAAEWD2Q6S77CG3J6G5WJIWPDXAQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:24.121673515 (2h26m27.137343125s) [upload=2343]: Next returned nil segment
Mar 10 18:37:24.121693564 (2h26m27.137363175s) [upload=2343]: Waiting for error group managing segments...
Mar 10 18:37:24.258918473 (2h26m27.274588073s) [noise=true , address=45.91.116.4:4943 , piece_id=EOL7JSKHMYZQPZVM3Q5GN2HJC65YKJWG3KTJMICUA4PLSIERLDZQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.259101868 (2h26m27.274771472s) [noise=true , address=79.127.219.41:4947 , piece_id=NI3ZGQNFOLUIRH3B4QB3FDXFLVNCYKGX24U4HROKCESWMXNKC54A, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.265791082 (2h26m27.281460686s) [noise=true , address=145.79.227.17:4000 , piece_id=AU4UBLJAXLH7JNNYKC3YB3UIJAAEWD2Q6S77CG3J6G5WJIWPDXAQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.277538368 (2h26m27.293207972s) [noise=true , address=45.91.116.13:4167 , piece_id=3GJVLC4HXSBZ6D3FNOYONOTF6ZN2YUKPI3N4XFX34E7TTT7KE77Q, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.283507633 (2h26m27.299177234s) [noise=true , address=45.91.116.10:4093 , piece_id=HUH66QBVZCXFN7BLJRADGU2H42MKXL3WIICJI6LAWQF2J3YXIGRQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.288254595 (2h26m27.303924202s) [noise=true , address=45.91.116.6:4999 , piece_id=DJDQ6BIBJESD4OKWTL6JIACEQ5LU4LROC3WNXEF6APEM734BOXZQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.650345388 (2h26m27.666014988s) [noise=true , address=167.17.72.68:4659 , piece_id=4LQBLKHK5RRKVVYFSPSMDWHWG2QPDHZNG2REAVL2PMWO3VZ36NAQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.658677204 (2h26m27.674346807s) [noise=true , address=167.17.72.69:4727 , piece_id=2PMBDX2OJJD33H3GCSQRJAPZMOV5II3WLZOY5YF4HKE7GYBQU6AQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.804520312 (2h26m27.820189912s) [noise=true , address=193.149.164.69:4081 , piece_id=6DNISIGMWMIMA6OICHGJUJOR6NNMXSCJI6B4GO6VKT4MCTMKG3HA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.805528476 (2h26m27.821198079s) [noise=true , address=193.149.164.68:4068 , piece_id=7WA3675HW4UPV7ZTNJEZIJDB7IIHFWLVHTSXOR5UMLQIXNHE7AXA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.810879435 (2h26m27.826549059s) [noise=true , address=193.149.164.67:4016 , piece_id=CITHWRKZJJGV4X3VG7CPQPWX2FPJDXU4QMSFMVE6ANNKAIICEUMA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.813793209 (2h26m27.829462813s) [noise=true , address=193.149.164.66:4978 , piece_id=UFBW2KYD7LUKXGBAOB4JOTNEKUE77KZSKFFJ3EFDGHPC7QKPAFXQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:38:23.155284138 (2h27m26.170953744s) [noise=true , address=77.90.31.12:4614 , piece_id=MLXXO4453CEWAG3G5WZZLAAX7LWUSFVVWQ3FXFKODALSAWRHLEFA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:12wKLBwTAdUVAQmQGcafdpUcjiDE89jxdptEwXoAvV6WXf2Xbc3, address:77.90.31.12:4614): protocol: rpc: dial tcp 77.90.31.12:4614: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155359187 (2h27m26.171028792s) [noise=true , address=77.90.31.17:4796 , piece_id=FZMZYUGLKY5FPBA3GEF7UVM5IBNO2KRUSEQCJSUVL2PRRN5AWOMQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1KK6y5NBuZovne8n3gMvAyDcZFasZxDzorvPeHe7CefoFs27UF, address:77.90.31.17:4796): protocol: write tcp 139.59.144.22:52670->77.90.31.17:4796: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155422072 (2h27m26.171091682s) [noise=true , address=77.90.31.15:4717 , piece_id=MLYGZT47MTTYGBLNTPA7DPM7NE6EQ6MPZYFW7KVDLFXR4MD2ORPA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:12ULANtxKNaJviAZFsmHjDQWjG4x2wfmjCwbQtHmn9LpnELmkPc, address:77.90.31.15:4717): protocol: rpc: dial tcp 77.90.31.15:4717: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155489065 (2h27m26.171158674s) [noise=true , address=77.90.31.13:4645 , piece_id=UXFW5W5TPZBMH6IC7AT5GN2ZDMZTNDJLFH7OMNKWN5JXKAT76SFA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: rpc: dial tcp 77.90.31.13:4645: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155521077 (2h27m26.171190679s) [noise=true , address=77.90.31.14:4673 , piece_id=XY5KYY3SODVHKX2EM2NJT6FRSDGSKNZFSLOKPP3T4VKC423TMQUA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1xMwYmwF8fukP4nNY3GNLdZZ3a4YqVr35poQf3Trq7AYUNF7sh, address:77.90.31.14:4673): protocol: rpc: dial tcp 77.90.31.14:4673: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155568492 (2h27m26.171238098s) [noise=true , address=77.90.31.19:4874 , piece_id=DC4C4HMX2TD6FTW5RPS3O55IHEQ6E7OR6VKKREVKVW7FAFAPZ2XA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:14J8ruKPB6P9YBLPhHpCrcLJt3WBYwNo3fGBeJ2Pdp6JfWs3W4, address:77.90.31.19:4874): protocol: rpc: dial tcp 77.90.31.19:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155592304 (2h27m26.171261905s) [seg_pos={0 0}, upload=2343]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Mar 10 18:38:23.260013959 (2h27m26.275683549s) [upload=2343]: Done uploading segments...
Mar 10 18:38:23.260057302 (2h27m26.275726903s) [upload=2343]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.272372518s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............................................X............] duration:
1.643928158s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:39:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.64395414s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.883098047s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:39:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.883118176s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................................................X...........] duration:
1.645844225s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:38:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.645870335s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
3.077100109s trace:
https://jaeger.dev.storj.io/trace/b68ffcae175905a
FAIL (2026-03-10 18:38:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
05104 (2h26m26.456874693s) [upload=2343]: Got next segment. Inline: false
Mar 10 18:37:23.441245955 (2h26m26.456915545s) [upload=2343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 18:37:23.441251057 (2h26m26.45692066s) [upload=2343]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 18:37:23.557618294 (2h26m26.573287881s) [seg_pos={0 0}, upload=2343]: Begin upload segment...
Mar 10 18:37:23.557695669 (2h26m26.573365278s) [noise=true , address=77.90.31.13:4645 , piece_id=UXFW5W5TPZBMH6IC7AT5GN2ZDMZTNDJLFH7OMNKWN5JXKAT76SFA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.558906475 (2h26m26.574576087s) [seg_pos={0 0}, upload=2343]: Done begin upload segment.
Mar 10 18:37:23.558987696 (2h26m26.57465731s) [noise=true , address=79.127.219.41:4947 , piece_id=NI3ZGQNFOLUIRH3B4QB3FDXFLVNCYKGX24U4HROKCESWMXNKC54A, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.559601912 (2h26m26.575271515s) [noise=true , address=77.90.31.19:4874 , piece_id=DC4C4HMX2TD6FTW5RPS3O55IHEQ6E7OR6VKKREVKVW7FAFAPZ2XA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.560517706 (2h26m26.57618731s) [noise=true , address=45.91.116.4:4943 , piece_id=EOL7JSKHMYZQPZVM3Q5GN2HJC65YKJWG3KTJMICUA4PLSIERLDZQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.561695181 (2h26m26.577364785s) [noise=true , address=77.90.31.14:4673 , piece_id=XY5KYY3SODVHKX2EM2NJT6FRSDGSKNZFSLOKPP3T4VKC423TMQUA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.562028023 (2h26m26.57769763s) [noise=true , address=193.149.164.68:4068 , piece_id=7WA3675HW4UPV7ZTNJEZIJDB7IIHFWLVHTSXOR5UMLQIXNHE7AXA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.562622388 (2h26m26.578292229s) [noise=true , address=45.91.116.6:4999 , piece_id=DJDQ6BIBJESD4OKWTL6JIACEQ5LU4LROC3WNXEF6APEM734BOXZQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.564570119 (2h26m26.580239738s) [noise=true , address=193.149.164.69:4081 , piece_id=6DNISIGMWMIMA6OICHGJUJOR6NNMXSCJI6B4GO6VKT4MCTMKG3HA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.564708737 (2h26m26.58037834s) [noise=true , address=167.17.72.68:4659 , piece_id=4LQBLKHK5RRKVVYFSPSMDWHWG2QPDHZNG2REAVL2PMWO3VZ36NAQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.566432612 (2h26m26.582102222s) [noise=true , address=167.17.72.69:4727 , piece_id=2PMBDX2OJJD33H3GCSQRJAPZMOV5II3WLZOY5YF4HKE7GYBQU6AQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.567575901 (2h26m26.583245509s) [noise=true , address=77.90.31.15:4717 , piece_id=MLYGZT47MTTYGBLNTPA7DPM7NE6EQ6MPZYFW7KVDLFXR4MD2ORPA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.567768288 (2h26m26.583437899s) [noise=true , address=193.149.164.67:4016 , piece_id=CITHWRKZJJGV4X3VG7CPQPWX2FPJDXU4QMSFMVE6ANNKAIICEUMA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.568437376 (2h26m26.584106976s) [noise=true , address=45.91.116.13:4167 , piece_id=3GJVLC4HXSBZ6D3FNOYONOTF6ZN2YUKPI3N4XFX34E7TTT7KE77Q, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.569626647 (2h26m26.585296258s) [noise=true , address=193.149.164.66:4978 , piece_id=UFBW2KYD7LUKXGBAOB4JOTNEKUE77KZSKFFJ3EFDGHPC7QKPAFXQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.570719385 (2h26m26.586388996s) [noise=true , address=45.91.116.10:4093 , piece_id=HUH66QBVZCXFN7BLJRADGU2H42MKXL3WIICJI6LAWQF2J3YXIGRQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.572104408 (2h26m26.587774016s) [noise=true , address=77.90.31.17:4796 , piece_id=FZMZYUGLKY5FPBA3GEF7UVM5IBNO2KRUSEQCJSUVL2PRRN5AWOMQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.573696158 (2h26m26.589365758s) [noise=true , address=77.90.31.12:4614 , piece_id=MLXXO4453CEWAG3G5WZZLAAX7LWUSFVVWQ3FXFKODALSAWRHLEFA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.575002207 (2h26m26.590671812s) [noise=true , address=145.79.227.17:4000 , piece_id=AU4UBLJAXLH7JNNYKC3YB3UIJAAEWD2Q6S77CG3J6G5WJIWPDXAQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:24.121673515 (2h26m27.137343125s) [upload=2343]: Next returned nil segment
Mar 10 18:37:24.121693564 (2h26m27.137363175s) [upload=2343]: Waiting for error group managing segments...
Mar 10 18:37:24.258918473 (2h26m27.274588073s) [noise=true , address=45.91.116.4:4943 , piece_id=EOL7JSKHMYZQPZVM3Q5GN2HJC65YKJWG3KTJMICUA4PLSIERLDZQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.259101868 (2h26m27.274771472s) [noise=true , address=79.127.219.41:4947 , piece_id=NI3ZGQNFOLUIRH3B4QB3FDXFLVNCYKGX24U4HROKCESWMXNKC54A, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.265791082 (2h26m27.281460686s) [noise=true , address=145.79.227.17:4000 , piece_id=AU4UBLJAXLH7JNNYKC3YB3UIJAAEWD2Q6S77CG3J6G5WJIWPDXAQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.277538368 (2h26m27.293207972s) [noise=true , address=45.91.116.13:4167 , piece_id=3GJVLC4HXSBZ6D3FNOYONOTF6ZN2YUKPI3N4XFX34E7TTT7KE77Q, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.283507633 (2h26m27.299177234s) [noise=true , address=45.91.116.10:4093 , piece_id=HUH66QBVZCXFN7BLJRADGU2H42MKXL3WIICJI6LAWQF2J3YXIGRQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.288254595 (2h26m27.303924202s) [noise=true , address=45.91.116.6:4999 , piece_id=DJDQ6BIBJESD4OKWTL6JIACEQ5LU4LROC3WNXEF6APEM734BOXZQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.650345388 (2h26m27.666014988s) [noise=true , address=167.17.72.68:4659 , piece_id=4LQBLKHK5RRKVVYFSPSMDWHWG2QPDHZNG2REAVL2PMWO3VZ36NAQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.658677204 (2h26m27.674346807s) [noise=true , address=167.17.72.69:4727 , piece_id=2PMBDX2OJJD33H3GCSQRJAPZMOV5II3WLZOY5YF4HKE7GYBQU6AQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.804520312 (2h26m27.820189912s) [noise=true , address=193.149.164.69:4081 , piece_id=6DNISIGMWMIMA6OICHGJUJOR6NNMXSCJI6B4GO6VKT4MCTMKG3HA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.805528476 (2h26m27.821198079s) [noise=true , address=193.149.164.68:4068 , piece_id=7WA3675HW4UPV7ZTNJEZIJDB7IIHFWLVHTSXOR5UMLQIXNHE7AXA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.810879435 (2h26m27.826549059s) [noise=true , address=193.149.164.67:4016 , piece_id=CITHWRKZJJGV4X3VG7CPQPWX2FPJDXU4QMSFMVE6ANNKAIICEUMA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.813793209 (2h26m27.829462813s) [noise=true , address=193.149.164.66:4978 , piece_id=UFBW2KYD7LUKXGBAOB4JOTNEKUE77KZSKFFJ3EFDGHPC7QKPAFXQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:38:23.155284138 (2h27m26.170953744s) [noise=true , address=77.90.31.12:4614 , piece_id=MLXXO4453CEWAG3G5WZZLAAX7LWUSFVVWQ3FXFKODALSAWRHLEFA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:12wKLBwTAdUVAQmQGcafdpUcjiDE89jxdptEwXoAvV6WXf2Xbc3, address:77.90.31.12:4614): protocol: rpc: dial tcp 77.90.31.12:4614: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155359187 (2h27m26.171028792s) [noise=true , address=77.90.31.17:4796 , piece_id=FZMZYUGLKY5FPBA3GEF7UVM5IBNO2KRUSEQCJSUVL2PRRN5AWOMQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1KK6y5NBuZovne8n3gMvAyDcZFasZxDzorvPeHe7CefoFs27UF, address:77.90.31.17:4796): protocol: write tcp 139.59.144.22:52670->77.90.31.17:4796: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155422072 (2h27m26.171091682s) [noise=true , address=77.90.31.15:4717 , piece_id=MLYGZT47MTTYGBLNTPA7DPM7NE6EQ6MPZYFW7KVDLFXR4MD2ORPA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:12ULANtxKNaJviAZFsmHjDQWjG4x2wfmjCwbQtHmn9LpnELmkPc, address:77.90.31.15:4717): protocol: rpc: dial tcp 77.90.31.15:4717: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155489065 (2h27m26.171158674s) [noise=true , address=77.90.31.13:4645 , piece_id=UXFW5W5TPZBMH6IC7AT5GN2ZDMZTNDJLFH7OMNKWN5JXKAT76SFA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: rpc: dial tcp 77.90.31.13:4645: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155521077 (2h27m26.171190679s) [noise=true , address=77.90.31.14:4673 , piece_id=XY5KYY3SODVHKX2EM2NJT6FRSDGSKNZFSLOKPP3T4VKC423TMQUA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1xMwYmwF8fukP4nNY3GNLdZZ3a4YqVr35poQf3Trq7AYUNF7sh, address:77.90.31.14:4673): protocol: rpc: dial tcp 77.90.31.14:4673: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155568492 (2h27m26.171238098s) [noise=true , address=77.90.31.19:4874 , piece_id=DC4C4HMX2TD6FTW5RPS3O55IHEQ6E7OR6VKKREVKVW7FAFAPZ2XA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:14J8ruKPB6P9YBLPhHpCrcLJt3WBYwNo3fGBeJ2Pdp6JfWs3W4, address:77.90.31.19:4874): protocol: rpc: dial tcp 77.90.31.19:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155592304 (2h27m26.171261905s) [seg_pos={0 0}, upload=2343]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Mar 10 18:38:23.260013959 (2h27m26.275683549s) [upload=2343]: Done uploading segments...
Mar 10 18:38:23.260057302 (2h27m26.275726903s) [upload=2343]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.077117662s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.966495726s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:38:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.966524754s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
3.946499522s trace:
https://jaeger.dev.storj.io/trace/b68ffcae175905a
FAIL (2026-03-10 18:38:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
05104 (2h26m26.456874693s) [upload=2343]: Got next segment. Inline: false
Mar 10 18:37:23.441245955 (2h26m26.456915545s) [upload=2343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 18:37:23.441251057 (2h26m26.45692066s) [upload=2343]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 18:37:23.557618294 (2h26m26.573287881s) [seg_pos={0 0}, upload=2343]: Begin upload segment...
Mar 10 18:37:23.557695669 (2h26m26.573365278s) [noise=true , address=77.90.31.13:4645 , piece_id=UXFW5W5TPZBMH6IC7AT5GN2ZDMZTNDJLFH7OMNKWN5JXKAT76SFA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.558906475 (2h26m26.574576087s) [seg_pos={0 0}, upload=2343]: Done begin upload segment.
Mar 10 18:37:23.558987696 (2h26m26.57465731s) [noise=true , address=79.127.219.41:4947 , piece_id=NI3ZGQNFOLUIRH3B4QB3FDXFLVNCYKGX24U4HROKCESWMXNKC54A, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.559601912 (2h26m26.575271515s) [noise=true , address=77.90.31.19:4874 , piece_id=DC4C4HMX2TD6FTW5RPS3O55IHEQ6E7OR6VKKREVKVW7FAFAPZ2XA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.560517706 (2h26m26.57618731s) [noise=true , address=45.91.116.4:4943 , piece_id=EOL7JSKHMYZQPZVM3Q5GN2HJC65YKJWG3KTJMICUA4PLSIERLDZQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.561695181 (2h26m26.577364785s) [noise=true , address=77.90.31.14:4673 , piece_id=XY5KYY3SODVHKX2EM2NJT6FRSDGSKNZFSLOKPP3T4VKC423TMQUA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.562028023 (2h26m26.57769763s) [noise=true , address=193.149.164.68:4068 , piece_id=7WA3675HW4UPV7ZTNJEZIJDB7IIHFWLVHTSXOR5UMLQIXNHE7AXA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.562622388 (2h26m26.578292229s) [noise=true , address=45.91.116.6:4999 , piece_id=DJDQ6BIBJESD4OKWTL6JIACEQ5LU4LROC3WNXEF6APEM734BOXZQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.564570119 (2h26m26.580239738s) [noise=true , address=193.149.164.69:4081 , piece_id=6DNISIGMWMIMA6OICHGJUJOR6NNMXSCJI6B4GO6VKT4MCTMKG3HA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.564708737 (2h26m26.58037834s) [noise=true , address=167.17.72.68:4659 , piece_id=4LQBLKHK5RRKVVYFSPSMDWHWG2QPDHZNG2REAVL2PMWO3VZ36NAQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.566432612 (2h26m26.582102222s) [noise=true , address=167.17.72.69:4727 , piece_id=2PMBDX2OJJD33H3GCSQRJAPZMOV5II3WLZOY5YF4HKE7GYBQU6AQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.567575901 (2h26m26.583245509s) [noise=true , address=77.90.31.15:4717 , piece_id=MLYGZT47MTTYGBLNTPA7DPM7NE6EQ6MPZYFW7KVDLFXR4MD2ORPA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.567768288 (2h26m26.583437899s) [noise=true , address=193.149.164.67:4016 , piece_id=CITHWRKZJJGV4X3VG7CPQPWX2FPJDXU4QMSFMVE6ANNKAIICEUMA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.568437376 (2h26m26.584106976s) [noise=true , address=45.91.116.13:4167 , piece_id=3GJVLC4HXSBZ6D3FNOYONOTF6ZN2YUKPI3N4XFX34E7TTT7KE77Q, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.569626647 (2h26m26.585296258s) [noise=true , address=193.149.164.66:4978 , piece_id=UFBW2KYD7LUKXGBAOB4JOTNEKUE77KZSKFFJ3EFDGHPC7QKPAFXQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.570719385 (2h26m26.586388996s) [noise=true , address=45.91.116.10:4093 , piece_id=HUH66QBVZCXFN7BLJRADGU2H42MKXL3WIICJI6LAWQF2J3YXIGRQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.572104408 (2h26m26.587774016s) [noise=true , address=77.90.31.17:4796 , piece_id=FZMZYUGLKY5FPBA3GEF7UVM5IBNO2KRUSEQCJSUVL2PRRN5AWOMQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.573696158 (2h26m26.589365758s) [noise=true , address=77.90.31.12:4614 , piece_id=MLXXO4453CEWAG3G5WZZLAAX7LWUSFVVWQ3FXFKODALSAWRHLEFA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.575002207 (2h26m26.590671812s) [noise=true , address=145.79.227.17:4000 , piece_id=AU4UBLJAXLH7JNNYKC3YB3UIJAAEWD2Q6S77CG3J6G5WJIWPDXAQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:24.121673515 (2h26m27.137343125s) [upload=2343]: Next returned nil segment
Mar 10 18:37:24.121693564 (2h26m27.137363175s) [upload=2343]: Waiting for error group managing segments...
Mar 10 18:37:24.258918473 (2h26m27.274588073s) [noise=true , address=45.91.116.4:4943 , piece_id=EOL7JSKHMYZQPZVM3Q5GN2HJC65YKJWG3KTJMICUA4PLSIERLDZQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.259101868 (2h26m27.274771472s) [noise=true , address=79.127.219.41:4947 , piece_id=NI3ZGQNFOLUIRH3B4QB3FDXFLVNCYKGX24U4HROKCESWMXNKC54A, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.265791082 (2h26m27.281460686s) [noise=true , address=145.79.227.17:4000 , piece_id=AU4UBLJAXLH7JNNYKC3YB3UIJAAEWD2Q6S77CG3J6G5WJIWPDXAQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.277538368 (2h26m27.293207972s) [noise=true , address=45.91.116.13:4167 , piece_id=3GJVLC4HXSBZ6D3FNOYONOTF6ZN2YUKPI3N4XFX34E7TTT7KE77Q, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.283507633 (2h26m27.299177234s) [noise=true , address=45.91.116.10:4093 , piece_id=HUH66QBVZCXFN7BLJRADGU2H42MKXL3WIICJI6LAWQF2J3YXIGRQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.288254595 (2h26m27.303924202s) [noise=true , address=45.91.116.6:4999 , piece_id=DJDQ6BIBJESD4OKWTL6JIACEQ5LU4LROC3WNXEF6APEM734BOXZQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.650345388 (2h26m27.666014988s) [noise=true , address=167.17.72.68:4659 , piece_id=4LQBLKHK5RRKVVYFSPSMDWHWG2QPDHZNG2REAVL2PMWO3VZ36NAQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.658677204 (2h26m27.674346807s) [noise=true , address=167.17.72.69:4727 , piece_id=2PMBDX2OJJD33H3GCSQRJAPZMOV5II3WLZOY5YF4HKE7GYBQU6AQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.804520312 (2h26m27.820189912s) [noise=true , address=193.149.164.69:4081 , piece_id=6DNISIGMWMIMA6OICHGJUJOR6NNMXSCJI6B4GO6VKT4MCTMKG3HA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.805528476 (2h26m27.821198079s) [noise=true , address=193.149.164.68:4068 , piece_id=7WA3675HW4UPV7ZTNJEZIJDB7IIHFWLVHTSXOR5UMLQIXNHE7AXA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.810879435 (2h26m27.826549059s) [noise=true , address=193.149.164.67:4016 , piece_id=CITHWRKZJJGV4X3VG7CPQPWX2FPJDXU4QMSFMVE6ANNKAIICEUMA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.813793209 (2h26m27.829462813s) [noise=true , address=193.149.164.66:4978 , piece_id=UFBW2KYD7LUKXGBAOB4JOTNEKUE77KZSKFFJ3EFDGHPC7QKPAFXQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:38:23.155284138 (2h27m26.170953744s) [noise=true , address=77.90.31.12:4614 , piece_id=MLXXO4453CEWAG3G5WZZLAAX7LWUSFVVWQ3FXFKODALSAWRHLEFA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:12wKLBwTAdUVAQmQGcafdpUcjiDE89jxdptEwXoAvV6WXf2Xbc3, address:77.90.31.12:4614): protocol: rpc: dial tcp 77.90.31.12:4614: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155359187 (2h27m26.171028792s) [noise=true , address=77.90.31.17:4796 , piece_id=FZMZYUGLKY5FPBA3GEF7UVM5IBNO2KRUSEQCJSUVL2PRRN5AWOMQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1KK6y5NBuZovne8n3gMvAyDcZFasZxDzorvPeHe7CefoFs27UF, address:77.90.31.17:4796): protocol: write tcp 139.59.144.22:52670->77.90.31.17:4796: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155422072 (2h27m26.171091682s) [noise=true , address=77.90.31.15:4717 , piece_id=MLYGZT47MTTYGBLNTPA7DPM7NE6EQ6MPZYFW7KVDLFXR4MD2ORPA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:12ULANtxKNaJviAZFsmHjDQWjG4x2wfmjCwbQtHmn9LpnELmkPc, address:77.90.31.15:4717): protocol: rpc: dial tcp 77.90.31.15:4717: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155489065 (2h27m26.171158674s) [noise=true , address=77.90.31.13:4645 , piece_id=UXFW5W5TPZBMH6IC7AT5GN2ZDMZTNDJLFH7OMNKWN5JXKAT76SFA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: rpc: dial tcp 77.90.31.13:4645: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155521077 (2h27m26.171190679s) [noise=true , address=77.90.31.14:4673 , piece_id=XY5KYY3SODVHKX2EM2NJT6FRSDGSKNZFSLOKPP3T4VKC423TMQUA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1xMwYmwF8fukP4nNY3GNLdZZ3a4YqVr35poQf3Trq7AYUNF7sh, address:77.90.31.14:4673): protocol: rpc: dial tcp 77.90.31.14:4673: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155568492 (2h27m26.171238098s) [noise=true , address=77.90.31.19:4874 , piece_id=DC4C4HMX2TD6FTW5RPS3O55IHEQ6E7OR6VKKREVKVW7FAFAPZ2XA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:14J8ruKPB6P9YBLPhHpCrcLJt3WBYwNo3fGBeJ2Pdp6JfWs3W4, address:77.90.31.19:4874): protocol: rpc: dial tcp 77.90.31.19:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155592304 (2h27m26.171261905s) [seg_pos={0 0}, upload=2343]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Mar 10 18:38:23.260013959 (2h27m26.275683549s) [upload=2343]: Done uploading segments...
Mar 10 18:38:23.260057302 (2h27m26.275726903s) [upload=2343]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.946518269s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.106616189s trace:
https://jaeger.dev.storj.io/trace/b68ffcae175905a
FAIL (2026-03-10 18:38:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
05104 (2h26m26.456874693s) [upload=2343]: Got next segment. Inline: false
Mar 10 18:37:23.441245955 (2h26m26.456915545s) [upload=2343]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 18:37:23.441251057 (2h26m26.45692066s) [upload=2343]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 18:37:23.557618294 (2h26m26.573287881s) [seg_pos={0 0}, upload=2343]: Begin upload segment...
Mar 10 18:37:23.557695669 (2h26m26.573365278s) [noise=true , address=77.90.31.13:4645 , piece_id=UXFW5W5TPZBMH6IC7AT5GN2ZDMZTNDJLFH7OMNKWN5JXKAT76SFA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.558906475 (2h26m26.574576087s) [seg_pos={0 0}, upload=2343]: Done begin upload segment.
Mar 10 18:37:23.558987696 (2h26m26.57465731s) [noise=true , address=79.127.219.41:4947 , piece_id=NI3ZGQNFOLUIRH3B4QB3FDXFLVNCYKGX24U4HROKCESWMXNKC54A, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.559601912 (2h26m26.575271515s) [noise=true , address=77.90.31.19:4874 , piece_id=DC4C4HMX2TD6FTW5RPS3O55IHEQ6E7OR6VKKREVKVW7FAFAPZ2XA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.560517706 (2h26m26.57618731s) [noise=true , address=45.91.116.4:4943 , piece_id=EOL7JSKHMYZQPZVM3Q5GN2HJC65YKJWG3KTJMICUA4PLSIERLDZQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.561695181 (2h26m26.577364785s) [noise=true , address=77.90.31.14:4673 , piece_id=XY5KYY3SODVHKX2EM2NJT6FRSDGSKNZFSLOKPP3T4VKC423TMQUA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.562028023 (2h26m26.57769763s) [noise=true , address=193.149.164.68:4068 , piece_id=7WA3675HW4UPV7ZTNJEZIJDB7IIHFWLVHTSXOR5UMLQIXNHE7AXA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.562622388 (2h26m26.578292229s) [noise=true , address=45.91.116.6:4999 , piece_id=DJDQ6BIBJESD4OKWTL6JIACEQ5LU4LROC3WNXEF6APEM734BOXZQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.564570119 (2h26m26.580239738s) [noise=true , address=193.149.164.69:4081 , piece_id=6DNISIGMWMIMA6OICHGJUJOR6NNMXSCJI6B4GO6VKT4MCTMKG3HA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.564708737 (2h26m26.58037834s) [noise=true , address=167.17.72.68:4659 , piece_id=4LQBLKHK5RRKVVYFSPSMDWHWG2QPDHZNG2REAVL2PMWO3VZ36NAQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.566432612 (2h26m26.582102222s) [noise=true , address=167.17.72.69:4727 , piece_id=2PMBDX2OJJD33H3GCSQRJAPZMOV5II3WLZOY5YF4HKE7GYBQU6AQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.567575901 (2h26m26.583245509s) [noise=true , address=77.90.31.15:4717 , piece_id=MLYGZT47MTTYGBLNTPA7DPM7NE6EQ6MPZYFW7KVDLFXR4MD2ORPA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.567768288 (2h26m26.583437899s) [noise=true , address=193.149.164.67:4016 , piece_id=CITHWRKZJJGV4X3VG7CPQPWX2FPJDXU4QMSFMVE6ANNKAIICEUMA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.568437376 (2h26m26.584106976s) [noise=true , address=45.91.116.13:4167 , piece_id=3GJVLC4HXSBZ6D3FNOYONOTF6ZN2YUKPI3N4XFX34E7TTT7KE77Q, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.569626647 (2h26m26.585296258s) [noise=true , address=193.149.164.66:4978 , piece_id=UFBW2KYD7LUKXGBAOB4JOTNEKUE77KZSKFFJ3EFDGHPC7QKPAFXQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.570719385 (2h26m26.586388996s) [noise=true , address=45.91.116.10:4093 , piece_id=HUH66QBVZCXFN7BLJRADGU2H42MKXL3WIICJI6LAWQF2J3YXIGRQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.572104408 (2h26m26.587774016s) [noise=true , address=77.90.31.17:4796 , piece_id=FZMZYUGLKY5FPBA3GEF7UVM5IBNO2KRUSEQCJSUVL2PRRN5AWOMQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.573696158 (2h26m26.589365758s) [noise=true , address=77.90.31.12:4614 , piece_id=MLXXO4453CEWAG3G5WZZLAAX7LWUSFVVWQ3FXFKODALSAWRHLEFA, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:23.575002207 (2h26m26.590671812s) [noise=true , address=145.79.227.17:4000 , piece_id=AU4UBLJAXLH7JNNYKC3YB3UIJAAEWD2Q6S77CG3J6G5WJIWPDXAQ, seg_pos={0 0}, upload=2343]: Uploading piece...
Mar 10 18:37:24.121673515 (2h26m27.137343125s) [upload=2343]: Next returned nil segment
Mar 10 18:37:24.121693564 (2h26m27.137363175s) [upload=2343]: Waiting for error group managing segments...
Mar 10 18:37:24.258918473 (2h26m27.274588073s) [noise=true , address=45.91.116.4:4943 , piece_id=EOL7JSKHMYZQPZVM3Q5GN2HJC65YKJWG3KTJMICUA4PLSIERLDZQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.259101868 (2h26m27.274771472s) [noise=true , address=79.127.219.41:4947 , piece_id=NI3ZGQNFOLUIRH3B4QB3FDXFLVNCYKGX24U4HROKCESWMXNKC54A, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.265791082 (2h26m27.281460686s) [noise=true , address=145.79.227.17:4000 , piece_id=AU4UBLJAXLH7JNNYKC3YB3UIJAAEWD2Q6S77CG3J6G5WJIWPDXAQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.277538368 (2h26m27.293207972s) [noise=true , address=45.91.116.13:4167 , piece_id=3GJVLC4HXSBZ6D3FNOYONOTF6ZN2YUKPI3N4XFX34E7TTT7KE77Q, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.283507633 (2h26m27.299177234s) [noise=true , address=45.91.116.10:4093 , piece_id=HUH66QBVZCXFN7BLJRADGU2H42MKXL3WIICJI6LAWQF2J3YXIGRQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.288254595 (2h26m27.303924202s) [noise=true , address=45.91.116.6:4999 , piece_id=DJDQ6BIBJESD4OKWTL6JIACEQ5LU4LROC3WNXEF6APEM734BOXZQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.650345388 (2h26m27.666014988s) [noise=true , address=167.17.72.68:4659 , piece_id=4LQBLKHK5RRKVVYFSPSMDWHWG2QPDHZNG2REAVL2PMWO3VZ36NAQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.658677204 (2h26m27.674346807s) [noise=true , address=167.17.72.69:4727 , piece_id=2PMBDX2OJJD33H3GCSQRJAPZMOV5II3WLZOY5YF4HKE7GYBQU6AQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.804520312 (2h26m27.820189912s) [noise=true , address=193.149.164.69:4081 , piece_id=6DNISIGMWMIMA6OICHGJUJOR6NNMXSCJI6B4GO6VKT4MCTMKG3HA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.805528476 (2h26m27.821198079s) [noise=true , address=193.149.164.68:4068 , piece_id=7WA3675HW4UPV7ZTNJEZIJDB7IIHFWLVHTSXOR5UMLQIXNHE7AXA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.810879435 (2h26m27.826549059s) [noise=true , address=193.149.164.67:4016 , piece_id=CITHWRKZJJGV4X3VG7CPQPWX2FPJDXU4QMSFMVE6ANNKAIICEUMA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:37:24.813793209 (2h26m27.829462813s) [noise=true , address=193.149.164.66:4978 , piece_id=UFBW2KYD7LUKXGBAOB4JOTNEKUE77KZSKFFJ3EFDGHPC7QKPAFXQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: <nil>
Mar 10 18:38:23.155284138 (2h27m26.170953744s) [noise=true , address=77.90.31.12:4614 , piece_id=MLXXO4453CEWAG3G5WZZLAAX7LWUSFVVWQ3FXFKODALSAWRHLEFA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:12wKLBwTAdUVAQmQGcafdpUcjiDE89jxdptEwXoAvV6WXf2Xbc3, address:77.90.31.12:4614): protocol: rpc: dial tcp 77.90.31.12:4614: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155359187 (2h27m26.171028792s) [noise=true , address=77.90.31.17:4796 , piece_id=FZMZYUGLKY5FPBA3GEF7UVM5IBNO2KRUSEQCJSUVL2PRRN5AWOMQ, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1KK6y5NBuZovne8n3gMvAyDcZFasZxDzorvPeHe7CefoFs27UF, address:77.90.31.17:4796): protocol: write tcp 139.59.144.22:52670->77.90.31.17:4796: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155422072 (2h27m26.171091682s) [noise=true , address=77.90.31.15:4717 , piece_id=MLYGZT47MTTYGBLNTPA7DPM7NE6EQ6MPZYFW7KVDLFXR4MD2ORPA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:12ULANtxKNaJviAZFsmHjDQWjG4x2wfmjCwbQtHmn9LpnELmkPc, address:77.90.31.15:4717): protocol: rpc: dial tcp 77.90.31.15:4717: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155489065 (2h27m26.171158674s) [noise=true , address=77.90.31.13:4645 , piece_id=UXFW5W5TPZBMH6IC7AT5GN2ZDMZTNDJLFH7OMNKWN5JXKAT76SFA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: rpc: dial tcp 77.90.31.13:4645: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155521077 (2h27m26.171190679s) [noise=true , address=77.90.31.14:4673 , piece_id=XY5KYY3SODVHKX2EM2NJT6FRSDGSKNZFSLOKPP3T4VKC423TMQUA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:1xMwYmwF8fukP4nNY3GNLdZZ3a4YqVr35poQf3Trq7AYUNF7sh, address:77.90.31.14:4673): protocol: rpc: dial tcp 77.90.31.14:4673: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155568492 (2h27m26.171238098s) [noise=true , address=77.90.31.19:4874 , piece_id=DC4C4HMX2TD6FTW5RPS3O55IHEQ6E7OR6VKKREVKVW7FAFAPZ2XA, seg_pos={0 0}, upload=2343]: Done uploading piece. err: ecclient: upload failed (node:14J8ruKPB6P9YBLPhHpCrcLJt3WBYwNo3fGBeJ2Pdp6JfWs3W4, address:77.90.31.19:4874): protocol: rpc: dial tcp 77.90.31.19:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 18:38:23.155592304 (2h27m26.171261905s) [seg_pos={0 0}, upload=2343]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Mar 10 18:38:23.260013959 (2h27m26.275683549s) [upload=2343]: Done uploading segments...
Mar 10 18:38:23.260057302 (2h27m26.275726903s) [upload=2343]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 1m0.106637306s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................................X..........] duration:
1.814767717s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:38:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.814790406s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.783638853s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:37:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.783681518s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.861407101s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:37:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.861435585s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................................X.........] duration:
1.734932362s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:36:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.734959194s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................................................X........] duration:
1.683628628s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:36:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.683713303s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.851982893s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:36:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.852000906s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................................X.......] duration:
1.82689159s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:35:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.826993606s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.83250653s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:35:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.832530072s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................................X......] duration:
2.055259549s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:34:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.055287941s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.841827936s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:34:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.841862745s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................................X.....] duration:
1.654457327s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:33:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.654486953s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
2.706963926s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:33:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.70699617s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................................................X....] duration:
1.652643992s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:33:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.652670998s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
3.028412202s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:33:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.028434885s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................................X...] duration:
1.642899919s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:32:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.642923797s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
3.362851707s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:32:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.362870147s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................................................X..] duration:
1.637828988s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:31:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.637861474s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.674640653s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:31:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.675088019s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................................X.] duration:
1.649807497s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:30:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.649891068s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.848832872s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:30:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.848861209s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.001346088s trace:
https://jaeger.dev.storj.io/trace/43a7d841b992769c
FAIL (2026-03-10 18:29:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001360813s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X] duration:
5.001537528s trace:
https://jaeger.dev.storj.io/trace/7210fe2bf339936d
FAIL (2026-03-10 18:29:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001558047s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
30 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..............................] duration:
695.847836ms trace:
https://jaeger.dev.storj.io/trace/319548f8548970fc
FAIL (2026-03-10 16:09:51 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 10 15:47:04.137097243 (1h22m48.645208037s) [upload=1312]: Starting upload Mar 10 15:47:04.137186416 (1h22m48.645297215s) [upload=1312]: Done starting upload Mar 10 15:47:04.137319788 (1h22m48.645430564s) [upload=1312]: Uploading segments... Mar 10 15:47:04.137851117 (1h22m48.645961913s) [upload=1312]: Got next segment. Inline: false Mar 10 15:47:04.137975296 (1h22m48.646086082s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 10 15:47:04.137982443 (1h22m48.646093229s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 10 15:47:04.181313804 (1h22m48.689424599s) [upload=1312]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 10 15:47:04.181351570 (1h22m48.689462364s) [upload=1312]: Done uploading segments... Mar 10 15:47:04.181433614 (1h22m48.6895444s) [upload=1312]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 695.87246ms
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................X..................................] duration:
1.587087613s trace:
https://jaeger.dev.storj.io/trace/594ebac882ea9689
FAIL (2026-03-10 16:09:47 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.587329749s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................X.................................] duration:
2.055108726s trace:
https://jaeger.dev.storj.io/trace/594ebac882ea9689
FAIL (2026-03-10 16:09:25 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.055136422s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................X................................] duration:
1.664454345s trace:
https://jaeger.dev.storj.io/trace/594ebac882ea9689
FAIL (2026-03-10 16:09:06 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.66448394s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
31 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.............................] duration:
592.212277ms trace:
https://jaeger.dev.storj.io/trace/319548f8548970fc
FAIL (2026-03-10 16:09:00 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 10 15:47:04.137097243 (1h22m48.645208037s) [upload=1312]: Starting upload Mar 10 15:47:04.137186416 (1h22m48.645297215s) [upload=1312]: Done starting upload Mar 10 15:47:04.137319788 (1h22m48.645430564s) [upload=1312]: Uploading segments... Mar 10 15:47:04.137851117 (1h22m48.645961913s) [upload=1312]: Got next segment. Inline: false Mar 10 15:47:04.137975296 (1h22m48.646086082s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 10 15:47:04.137982443 (1h22m48.646093229s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 10 15:47:04.181313804 (1h22m48.689424599s) [upload=1312]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 10 15:47:04.181351570 (1h22m48.689462364s) [upload=1312]: Done uploading segments... Mar 10 15:47:04.181433614 (1h22m48.6895444s) [upload=1312]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 592.24013ms
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............................X...............................] duration:
1.777646847s trace:
https://jaeger.dev.storj.io/trace/594ebac882ea9689
FAIL (2026-03-10 16:08:28 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.777670778s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
32 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX............................] duration:
692.723953ms trace:
https://jaeger.dev.storj.io/trace/319548f8548970fc
FAIL (2026-03-10 16:07:53 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 10 15:47:04.137097243 (1h22m48.645208037s) [upload=1312]: Starting upload Mar 10 15:47:04.137186416 (1h22m48.645297215s) [upload=1312]: Done starting upload Mar 10 15:47:04.137319788 (1h22m48.645430564s) [upload=1312]: Uploading segments... Mar 10 15:47:04.137851117 (1h22m48.645961913s) [upload=1312]: Got next segment. Inline: false Mar 10 15:47:04.137975296 (1h22m48.646086082s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 10 15:47:04.137982443 (1h22m48.646093229s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 10 15:47:04.181313804 (1h22m48.689424599s) [upload=1312]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 10 15:47:04.181351570 (1h22m48.689462364s) [upload=1312]: Done uploading segments... Mar 10 15:47:04.181433614 (1h22m48.6895444s) [upload=1312]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 692.753837ms
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............................X..............................] duration:
1.684083359s trace:
https://jaeger.dev.storj.io/trace/594ebac882ea9689
FAIL (2026-03-10 16:07:21 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.684114403s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............................X.............................] duration:
1.722744321s trace:
https://jaeger.dev.storj.io/trace/594ebac882ea9689
FAIL (2026-03-10 16:07:03 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.722940781s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............................X............................] duration:
1.613797082s trace:
https://jaeger.dev.storj.io/trace/594ebac882ea9689
FAIL (2026-03-10 16:06:44 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.613822843s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
33 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...........................] duration:
637.630039ms trace:
https://jaeger.dev.storj.io/trace/319548f8548970fc
FAIL (2026-03-10 16:06:41 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 10 15:47:04.137097243 (1h22m48.645208037s) [upload=1312]: Starting upload Mar 10 15:47:04.137186416 (1h22m48.645297215s) [upload=1312]: Done starting upload Mar 10 15:47:04.137319788 (1h22m48.645430564s) [upload=1312]: Uploading segments... Mar 10 15:47:04.137851117 (1h22m48.645961913s) [upload=1312]: Got next segment. Inline: false Mar 10 15:47:04.137975296 (1h22m48.646086082s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 10 15:47:04.137982443 (1h22m48.646093229s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 10 15:47:04.181313804 (1h22m48.689424599s) [upload=1312]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 10 15:47:04.181351570 (1h22m48.689462364s) [upload=1312]: Done uploading segments... Mar 10 15:47:04.181433614 (1h22m48.6895444s) [upload=1312]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 637.657675ms
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................................X...........................] duration:
1.589129182s trace:
https://jaeger.dev.storj.io/trace/594ebac882ea9689
FAIL (2026-03-10 16:06:26 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.589475356s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................X..........................] duration:
1.616181587s trace:
https://jaeger.dev.storj.io/trace/594ebac882ea9689
FAIL (2026-03-10 16:05:48 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.616209573s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
34 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..........................] duration:
576.106184ms trace:
https://jaeger.dev.storj.io/trace/319548f8548970fc
FAIL (2026-03-10 16:05:31 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 10 15:47:04.137097243 (1h22m48.645208037s) [upload=1312]: Starting upload Mar 10 15:47:04.137186416 (1h22m48.645297215s) [upload=1312]: Done starting upload Mar 10 15:47:04.137319788 (1h22m48.645430564s) [upload=1312]: Uploading segments... Mar 10 15:47:04.137851117 (1h22m48.645961913s) [upload=1312]: Got next segment. Inline: false Mar 10 15:47:04.137975296 (1h22m48.646086082s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 10 15:47:04.137982443 (1h22m48.646093229s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 10 15:47:04.181313804 (1h22m48.689424599s) [upload=1312]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 10 15:47:04.181351570 (1h22m48.689462364s) [upload=1312]: Done uploading segments... Mar 10 15:47:04.181433614 (1h22m48.6895444s) [upload=1312]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 576.13465ms
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
35 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.........................] duration:
670.929821ms trace:
https://jaeger.dev.storj.io/trace/319548f8548970fc
FAIL (2026-03-10 16:05:06 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 10 15:47:04.137097243 (1h22m48.645208037s) [upload=1312]: Starting upload Mar 10 15:47:04.137186416 (1h22m48.645297215s) [upload=1312]: Done starting upload Mar 10 15:47:04.137319788 (1h22m48.645430564s) [upload=1312]: Uploading segments... Mar 10 15:47:04.137851117 (1h22m48.645961913s) [upload=1312]: Got next segment. Inline: false Mar 10 15:47:04.137975296 (1h22m48.646086082s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 10 15:47:04.137982443 (1h22m48.646093229s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 10 15:47:04.181313804 (1h22m48.689424599s) [upload=1312]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 10 15:47:04.181351570 (1h22m48.689462364s) [upload=1312]: Done uploading segments... Mar 10 15:47:04.181433614 (1h22m48.6895444s) [upload=1312]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 670.968082ms
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................X.........................] duration:
1.622211747s trace:
https://jaeger.dev.storj.io/trace/594ebac882ea9689
FAIL (2026-03-10 16:04:34 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.622304946s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
36 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX........................] duration:
624.181794ms trace:
https://jaeger.dev.storj.io/trace/319548f8548970fc
FAIL (2026-03-10 16:03:59 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 10 15:47:04.137097243 (1h22m48.645208037s) [upload=1312]: Starting upload Mar 10 15:47:04.137186416 (1h22m48.645297215s) [upload=1312]: Done starting upload Mar 10 15:47:04.137319788 (1h22m48.645430564s) [upload=1312]: Uploading segments... Mar 10 15:47:04.137851117 (1h22m48.645961913s) [upload=1312]: Got next segment. Inline: false Mar 10 15:47:04.137975296 (1h22m48.646086082s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 10 15:47:04.137982443 (1h22m48.646093229s) [upload=1312]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 10 15:47:04.181313804 (1h22m48.689424599s) [upload=1312]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 10 15:47:04.181351570 (1h22m48.689462364s) [upload=1312]: Done uploading segments... Mar 10 15:47:04.181433614 (1h22m48.6895444s) [upload=1312]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 624.205022ms
core
uplink
uk-select-1
uk1-placement-17
|