Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............XX...........................................] duration:
2.262220189s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:06:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.262259095s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..................XX........................................] duration:
2.129253772s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:06:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.129278475s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [................XX..........................................] duration:
2.293830729s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:06:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.293871197s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
3.584229785s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:06:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.584273879s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
3.597519824s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:05:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.597550091s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...................XX.......................................] duration:
1.652490047s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:05:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.652554528s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................XX.........................................] duration:
2.469453491s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:05:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.469537403s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
3.619144997s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:04:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.619171944s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................XX........................................] duration:
2.260267347s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:04:21 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.260314447s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [....................XX......................................] duration:
1.701575665s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:04:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.701677997s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.....................XX.....................................] duration:
2.057926857s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:04:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.057946139s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
3.342684043s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:03:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.342720817s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................XX.......................................] duration:
2.334248314s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:03:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.334276889s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
3.069895358s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:03:19 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.069930691s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [......................XX....................................] duration:
1.692017844s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:02:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.692279815s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
3.414646242s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:02:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.41467584s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................XX......................................] duration:
2.264914909s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:02:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.26494598s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.......................XX...................................] duration:
2.103994636s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:02:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.104021266s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [........................XX..................................] duration:
1.743522627s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:01:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.74363904s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
3.386933692s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:01:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.387037202s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration:
2.291382853s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:01:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.291409308s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................XX....................................] duration:
2.406004282s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:00:57 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.406118248s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.........................XX.................................] duration:
1.788861913s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:00:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.788895883s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
3.056224415s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:00:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.056261755s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................XX...................................] duration:
2.344365099s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 00:59:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.344478907s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..........................XX................................] duration:
1.592707216s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 00:59:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.592826498s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
3.360227055s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 00:59:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.360256109s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................XX..................................] duration:
2.515355437s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 00:58:46 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.515383005s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...........................XX...............................] duration:
2.126973348s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 00:58:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.127003222s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................XX.................................] duration:
2.382930743s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 00:58:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.382996881s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
3.46473004s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 00:57:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.464759453s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [............................XX..............................] duration:
1.757102457s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 00:57:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.75712612s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................XX................................] duration:
2.538059175s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 00:57:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.538104446s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
3.632871313s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 00:56:52 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.632911303s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.............................XX.............................] duration:
1.664149947s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 00:56:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.664182359s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
3.01338203s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 00:56:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.013481986s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........................XX...............................] duration:
2.498259198s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 00:56:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.498293872s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [............................XX..............................] duration:
2.387439855s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 00:55:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.387472099s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
3.493911328s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 00:55:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.49394536s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..............................XX............................] duration:
1.742832875s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 00:55:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.742860725s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
3.904534465s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 00:55:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.904568699s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............................XX.............................] duration:
2.363505948s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 00:54:45 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.363535883s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...............................XX...........................] duration:
1.734680366s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 00:54:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.734710375s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [................................XX..........................] duration:
1.71129445s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 00:54:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.711331563s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
3.535283002s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 00:53:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.535341221s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............................XX............................] duration:
2.489580018s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 00:53:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.489610248s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.................................XX.........................] duration:
1.735852692s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 00:53:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.735884714s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
3.678736624s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 00:53:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.678769993s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............................XX...........................] duration:
2.375590496s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 00:53:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.375756028s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [................................XX..........................] duration:
2.341937959s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 00:52:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.341974202s
core
uplink
us-select-1
|