Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X...X..........] duration: 2.318985965s FAIL (2025-10-07 10:20:41 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.319017573s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.351147144s FAIL (2025-10-07 10:20:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.351188033s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X...X.........] duration: 2.112081541s FAIL (2025-10-07 10:19:41 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.112143603s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X...X........] duration: 2.117227304s FAIL (2025-10-07 10:18:56 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.117273322s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.543346822s FAIL (2025-10-07 10:18:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.543401967s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.448767785s FAIL (2025-10-07 10:18:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.448794625s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................X...X.......] duration: 1.904971832s FAIL (2025-10-07 10:17:47 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.905011233s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X...X......] duration: 2.173811815s FAIL (2025-10-07 10:17:13 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.173851123s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.572985381s FAIL (2025-10-07 10:17:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.573016798s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X...X.....] duration: 2.201477293s FAIL (2025-10-07 10:16:32 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.201518533s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.353096028s FAIL (2025-10-07 10:16:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.353136722s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X...X....] duration: 2.496488187s FAIL (2025-10-07 10:15:50 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.496524293s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.367747841s FAIL (2025-10-07 10:15:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.36777926s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X...X...] duration: 2.210875043s FAIL (2025-10-07 10:14:56 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.210922203s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.389024797s FAIL (2025-10-07 10:14:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.389145879s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X...X..] duration: 2.186670129s FAIL (2025-10-07 10:14:14 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.186710475s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 22.321153678s FAIL (2025-10-07 10:13:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 22.321177734s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X...X.] duration: 22.033912386s FAIL (2025-10-07 10:13:35 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 22.033946879s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 22.537859577s FAIL (2025-10-07 10:13:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 22.537902247s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 59.463564893s FAIL (2025-10-07 10:13:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 10:12:07.306656774 (332h41m55.110121353s) [upload=292729]: Starting upload Oct 7 10:12:07.306794557 (332h41m55.110259153s) [upload=292729]: Done starting upload Oct 7 10:12:07.306959891 (332h41m55.110424468s) [upload=292729]: Uploading segments... Oct 7 10:12:07.307411836 (332h41m55.110876417s) [upload=292729]: Got next segment. Inline: false Oct 7 10:12:07.307485009 (332h41m55.110949602s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 10:12:07.307492302 (332h41m55.110956903s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 10:12:07.540963169 (332h41m55.344427762s) [seg_pos={0 0}, upload=292729]: Begin upload segment... Oct 7 10:12:07.541037655 (332h41m55.344502238s) [seg_pos={0 0}, upload=292729]: Done begin upload segment. Oct 7 10:12:07.541074187 (332h41m55.344538809s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541090713 (332h41m55.344555316s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.541784911 (332h41m55.345249515s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542309311 (332h41m55.345773916s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.542796975 (332h41m55.346261578s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543247199 (332h41m55.346711802s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.543739269 (332h41m55.347203872s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544225902 (332h41m55.347690502s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.544709310 (332h41m55.34817391s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.545163687 (332h41m55.348628289s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546029033 (332h41m55.349493633s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.546617358 (332h41m55.350081961s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.547493041 (332h41m55.350957635s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548088795 (332h41m55.351553395s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548847836 (332h41m55.35231244s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:07.548913768 (332h41m55.352378353s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Uploading piece... Oct 7 10:12:08.170536647 (332h41m55.974001241s) [upload=292729]: Next returned nil segment Oct 7 10:12:08.170559840 (332h41m55.974024475s) [upload=292729]: Waiting for error group managing segments... Oct 7 10:12:08.304430668 (332h41m56.107895265s) [noise=true , address=45.91.116.7:4009 , piece_id=PW6G7IKWGPLO4LT5KLIBIBNZFNDTL5JIEKEWTSF5PSJW55RIXT3Q, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.305363185 (332h41m56.108827777s) [noise=true , address=45.91.116.14:4190 , piece_id=2JHBABDWZZ65DDRBCTKPXD4IRXPSBHLS5KNF6H4KDCSOAL3HZLHA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.306700170 (332h41m56.110164754s) [noise=true , address=45.91.116.14:4191 , piece_id=V4YLZTE5FSJPIUPGDQ3K6OHEG42Z4TO2NSJAQVSJ3FIOFAEWRTIA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.307067518 (332h41m56.110532104s) [noise=true , address=45.91.116.12:4225 , piece_id=NYILBPM7TDFZHL4KZY5Z7SAW2UBBVIQ673OUNSJL6RNFAGD75RUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.308741631 (332h41m56.112206216s) [noise=true , address=45.91.116.4:4940 , piece_id=BSQJ3FL5STCWA3XHR53DPPB22KTIF2IONAW4CWEZSL6POZCZZZUQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.309537153 (332h41m56.113001742s) [noise=true , address=45.91.116.11:4118 , piece_id=Y4OOJXNAXVQAYFMA4DNPS3KK3CG2G72HFPWFQJCOHFLA52K5V4WA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.310340574 (332h41m56.113805153s) [noise=true , address=45.91.116.5:4297 , piece_id=5T5RVW3DRGCFHCWTV7VJVQN3URPAFM6QA6QNXFX4OBIRSR2L5NFQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.311182954 (332h41m56.11464754s) [noise=true , address=45.91.116.14:4189 , piece_id=WUG46J5L64X4OGQRMKQQEYFYAUH3INHXSARHCAIG4UJG3OZMJLBQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.313196428 (332h41m56.116661024s) [noise=true , address=45.91.116.13:4155 , piece_id=JWPKUVWT7W77RVD6KTHBLTAZX4POMNSYQCHGFR5TPVTIYCERFGUA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.318452659 (332h41m56.121917244s) [noise=true , address=145.79.227.16:4912 , piece_id=Y7SMHQVAEKVN2HRMIVZ7BYOMTZASGVJSBBN4IJFVJXH4YI2AHWYQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328737265 (332h41m56.132201846s) [noise=true , address=145.79.227.7:4295 , piece_id=UR47ZJ5GB65RH7Y2SQSXCXTH4GFPZ6LOYX24R4EP6KLDG5MRAVSQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.328934722 (332h41m56.132399301s) [noise=true , address=145.79.227.15:4876 , piece_id=LPWSL6B3FZOP7RK4WWZOEYAYXAQKXP6B4MVANTER4V22Y7XAFMJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.329976841 (332h41m56.133441425s) [noise=true , address=145.79.227.12:4767 , piece_id=6UNBSGDTBDP35VTEXRCZV2TBXHP2KQRD3CWBMD4T5C5J7R25MWWQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: <nil> Oct 7 10:12:08.330015374 (332h41m56.133479965s) [seg_pos={0 0}, upload=292729]: Segment reached optimal threshold of 13 pieces. Oct 7 10:12:08.330183190 (332h41m56.133647782s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330193562 (332h41m56.133658159s) [noise=true , address=145.79.227.17:4971 , piece_id=OB62T7NSEQG3C74WBTY22WFIVN7XNIYRPBMUVPMANACXAESPAQGQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330288373 (332h41m56.133752965s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330294885 (332h41m56.133759477s) [noise=true , address=145.79.227.11:4729 , piece_id=JQMMV2534IWEDOG3L3IH22OX4G2ATYYET7QYRIKFRRA5GGTAPGQA, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330452305 (332h41m56.13391691s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Oct 7 10:12:08.330464871 (332h41m56.133929473s) [noise=true , address=145.79.227.15:4875 , piece_id=Y3KC6OC6YPRWQX6CKMSUQ2MYBJOFMBVV5RHNN2BRI3ID5TY52ZJQ, seg_pos={0 0}, upload=292729]: Long tail cancellation due to optimal threshold Oct 7 10:12:08.330492924 (332h41m56.133957517s) [seg_pos={0 0}, upload=292729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 10:12:08.330636956 (332h41m56.134101547s) [upload=292729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 10:12:08.330650493 (332h41m56.134115078s) [upload=292729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 10:13:06.461738970 (332h42m54.265203553s) [upload=292729]: Done uploading segments... Oct 7 10:13:06.461768088 (332h42m54.265232682s) [upload=292729]: Upload finished. err: metaclient: context deadline exceeded Duration: 59.463655381s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................X...X] duration: 30.000790599s FAIL (2025-10-07 10:12:38 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000814683s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.928351042s FAIL (2025-10-07 10:11:20 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.928384262s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.550570347s FAIL (2025-10-07 10:11:01 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.550607537s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.946902391s FAIL (2025-10-07 10:09:48 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.946929918s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000636378s FAIL (2025-10-07 10:09:45 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 30.000658518s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.127738907s FAIL (2025-10-07 09:43:43 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.127768697s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.400066434s FAIL (2025-10-07 09:43:05 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.400094826s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.033929558s FAIL (2025-10-07 09:41:54 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.03396082s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.178534774s FAIL (2025-10-07 09:41:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.178567875s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.155102101s FAIL (2025-10-07 09:41:36 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.155138679s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.289097758s FAIL (2025-10-07 09:40:54 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.289150443s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.56134026s FAIL (2025-10-07 09:40:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.561370252s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.07723816s FAIL (2025-10-07 09:40:41 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.077275693s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.123042014s FAIL (2025-10-07 09:40:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.123070638s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.213155424s FAIL (2025-10-07 09:39:40 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.213255307s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.193085687s FAIL (2025-10-07 09:39:36 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.193140867s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.080402466s FAIL (2025-10-07 09:39:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.080430014s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.183597428s FAIL (2025-10-07 09:39:16 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.183626437s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.078715818s FAIL (2025-10-07 09:39:15 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.078748554s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.17981546s FAIL (2025-10-07 09:38:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.179892624s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.141194093s FAIL (2025-10-07 09:38:25 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.141225006s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.094194586s FAIL (2025-10-07 09:38:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.094220561s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 894.571331ms FAIL (2025-10-07 09:38:11 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 894.601977ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 866.79356ms FAIL (2025-10-07 09:37:42 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 866.824575ms core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.405602731s FAIL (2025-10-07 09:37:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.405626436s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.178273154s FAIL (2025-10-07 09:37:13 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.178308572s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.089257846s FAIL (2025-10-07 09:36:45 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.089290694s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.434679261s FAIL (2025-10-07 09:36:43 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.434712369s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 1.120040779s FAIL (2025-10-07 09:36:26 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.120069366s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.305208798s FAIL (2025-10-07 09:36:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.305239485s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)