Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXXXXXXXXXXXXX.X] duration:
5.001009561s trace:
https://jaeger.dev.storj.io/trace/7cfa1932860578be
FAIL (2026-03-18 21:39:36 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001032236s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [......................................................XX....] duration:
929.990023ms trace:
https://jaeger.dev.storj.io/trace/94bb3ca45ba55fa
FAIL (2026-03-18 21:39:13 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 930.021872ms
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
42 out of 60 is failed (threshold=6) [.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXXXXXXXXXXXXX.] duration:
2.174316916s trace:
https://jaeger.dev.storj.io/trace/468bbab930a6aba3
FAIL (2026-03-18 21:38:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.174352422s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......................................................XX...] duration:
1.172979813s trace:
https://jaeger.dev.storj.io/trace/94bb3ca45ba55fa
FAIL (2026-03-18 21:38:43 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 1.173015287s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................................................XX...] duration:
2.497672s trace:
https://jaeger.dev.storj.io/trace/1fea1be3553256d8
FAIL (2026-03-18 21:38:30 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 18 21:37:13.407226914 (197h26m16.422896563s) [upload=189623]: Starting upload
Mar 18 21:37:13.407304870 (197h26m16.422974473s) [upload=189623]: Done starting upload
Mar 18 21:37:13.407540972 (197h26m16.423210564s) [upload=189623]: Uploading segments...
Mar 18 21:37:13.408026524 (197h26m16.423696147s) [upload=189623]: Got next segment. Inline: false
Mar 18 21:37:13.408123553 (197h26m16.423793153s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 18 21:37:13.408130704 (197h26m16.423800304s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 18 21:37:21.655023605 (197h26m24.670693205s) [seg_pos={0 0}, upload=189623]: Begin upload segment...
Mar 18 21:37:21.655094469 (197h26m24.670764059s) [seg_pos={0 0}, upload=189623]: Done begin upload segment.
Mar 18 21:37:21.655129337 (197h26m24.67079894s) [noise=true , address=145.79.227.4:4189 , piece_id=NMJ2X5ZX7D3U7KRZZYC2MC4XAAJNPQAZAWHLLBY2HNXSZCKCQCUQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.656236260 (197h26m24.671905872s) [noise=true , address=145.79.227.37:4566 , piece_id=CDLXWWDZWSXAUILEWL34GDA6ZNMYDYCETMQ5DDA474H4K4JGA6NQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.656865510 (197h26m24.672535113s) [noise=true , address=145.79.227.20:4439 , piece_id=3FWVMDZ7DLJV652FESYLUQS6CRTDL4ESGMHWEATRSFTBY2B6LTLA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.657306588 (197h26m24.672976189s) [noise=true , address=145.79.227.35:4486 , piece_id=RSLN2AZ2HSBPPU7XTUASMYU2Y2PFZN4YW6ZMPIOJNZTEWKWEQXJA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.657690602 (197h26m24.673360202s) [noise=true , address=77.90.31.19:4845 , piece_id=53YIOMVMGMQYUKUQBWWDWLUI22FSBQEKBIGZTCPADBO2XRONWTGA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.658444416 (197h26m24.674114022s) [noise=true , address=145.79.227.8:4335 , piece_id=U3ICUMTLPLHULI57IQ3V3H4VVEEAXSMQ4GKVFXZOUYWRYN6O2TYA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.659018578 (197h26m24.674688178s) [noise=true , address=145.79.227.15:4877 , piece_id=BQJN5RVMLYB7IIX3IFHYVFN3D2A5GYPV5L3Q6FWJ3VKVWH4NZWRA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.659604237 (197h26m24.675273838s) [noise=true , address=145.79.227.26:4657 , piece_id=O6UZJ7OLIA4BJQ6IRCI5ERXO3IURKV3GLJVG476KSPMRVQGY7LYA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.660277318 (197h26m24.675946921s) [noise=true , address=145.79.227.46:4601 , piece_id=3SWEFW5XZ3DZLPLKJZE3NBRRJCDZ7F7TDA5UXM7QBJUOWZTVPUPQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.660830355 (197h26m24.67649996s) [noise=true , address=45.91.116.7:4012 , piece_id=2JJXJQZO47ZTD7PFKABUENRESMEJMJNKAYCV6E5PKPAPO3IOKNOA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.661333927 (197h26m24.677003527s) [noise=true , address=145.79.227.3:4149 , piece_id=O4PG4M6HCKSRJZMO6KHNSFCRF5JRD2RWLNOMQGPGN2BS6TN2B2RQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.664022598 (197h26m24.679692201s) [noise=true , address=145.79.227.18:4011 , piece_id=NLH7AVODIX6ICBEYYN5JXGGLWLVOXZVYA3XWEOGUEQCW4HZTHEFQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.666068628 (197h26m24.681738234s) [noise=true , address=145.79.227.48:4306 , piece_id=FOUGOBCQYYPKA45WADXNNU4CRYNJDWVFVIFBF6VRKINXKAVXNBTQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.667030758 (197h26m24.682700369s) [noise=true , address=77.90.31.14:4663 , piece_id=IWCMYCJXN32SUB2CBFCV5TVGBL6WBISPBAZCLONZ5YDM4CJ46ZRQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.671055812 (197h26m24.686725418s) [noise=true , address=145.79.227.39:4018 , piece_id=DBAKT7XQX5IZVLEXDY63HP4NXGZVF6RTFGTNSCYKNX6EFAWVJTKQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.677611108 (197h26m24.693280721s) [noise=true , address=145.79.227.33:4874 , piece_id=NLQ737OMRU53WHBROA6TWFT4JU2RWL6AFGAVFFNWLHHEHMHFU5LQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:22.277424525 (197h26m25.293094127s) [upload=189623]: Next returned nil segment
Mar 18 21:37:22.277443392 (197h26m25.293113004s) [upload=189623]: Waiting for error group managing segments...
Mar 18 21:37:22.414377107 (197h26m25.430046707s) [noise=true , address=145.79.227.48:4306 , piece_id=FOUGOBCQYYPKA45WADXNNU4CRYNJDWVFVIFBF6VRKINXKAVXNBTQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.418771029 (197h26m25.434440635s) [noise=true , address=145.79.227.39:4018 , piece_id=DBAKT7XQX5IZVLEXDY63HP4NXGZVF6RTFGTNSCYKNX6EFAWVJTKQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.420950984 (197h26m25.436620574s) [noise=true , address=145.79.227.8:4335 , piece_id=U3ICUMTLPLHULI57IQ3V3H4VVEEAXSMQ4GKVFXZOUYWRYN6O2TYA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.421003817 (197h26m25.436673414s) [noise=true , address=145.79.227.4:4189 , piece_id=NMJ2X5ZX7D3U7KRZZYC2MC4XAAJNPQAZAWHLLBY2HNXSZCKCQCUQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.421987949 (197h26m25.437657549s) [noise=true , address=145.79.227.18:4011 , piece_id=NLH7AVODIX6ICBEYYN5JXGGLWLVOXZVYA3XWEOGUEQCW4HZTHEFQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.422142469 (197h26m25.437812071s) [noise=true , address=145.79.227.26:4657 , piece_id=O6UZJ7OLIA4BJQ6IRCI5ERXO3IURKV3GLJVG476KSPMRVQGY7LYA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.425641685 (197h26m25.441311285s) [noise=true , address=145.79.227.20:4439 , piece_id=3FWVMDZ7DLJV652FESYLUQS6CRTDL4ESGMHWEATRSFTBY2B6LTLA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.426101126 (197h26m25.44177072s) [noise=true , address=145.79.227.3:4149 , piece_id=O4PG4M6HCKSRJZMO6KHNSFCRF5JRD2RWLNOMQGPGN2BS6TN2B2RQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.427039270 (197h26m25.44270887s) [noise=true , address=145.79.227.15:4877 , piece_id=BQJN5RVMLYB7IIX3IFHYVFN3D2A5GYPV5L3Q6FWJ3VKVWH4NZWRA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.427616837 (197h26m25.443286441s) [noise=true , address=145.79.227.46:4601 , piece_id=3SWEFW5XZ3DZLPLKJZE3NBRRJCDZ7F7TDA5UXM7QBJUOWZTVPUPQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.431130437 (197h26m25.446800038s) [noise=true , address=45.91.116.7:4012 , piece_id=2JJXJQZO47ZTD7PFKABUENRESMEJMJNKAYCV6E5PKPAPO3IOKNOA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.431183849 (197h26m25.446853442s) [noise=true , address=145.79.227.33:4874 , piece_id=NLQ737OMRU53WHBROA6TWFT4JU2RWL6AFGAVFFNWLHHEHMHFU5LQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.456250331 (197h26m25.471919937s) [noise=true , address=145.79.227.35:4486 , piece_id=RSLN2AZ2HSBPPU7XTUASMYU2Y2PFZN4YW6ZMPIOJNZTEWKWEQXJA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.456277546 (197h26m25.47194715s) [seg_pos={0 0}, upload=189623]: Satisfied cohort requirements after 13 pieces
Mar 18 21:37:22.456632071 (197h26m25.472301676s) [noise=true , address=77.90.31.14:4663 , piece_id=IWCMYCJXN32SUB2CBFCV5TVGBL6WBISPBAZCLONZ5YDM4CJ46ZRQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Mar 18 21:37:22.456649964 (197h26m25.47231957s) [noise=true , address=77.90.31.14:4663 , piece_id=IWCMYCJXN32SUB2CBFCV5TVGBL6WBISPBAZCLONZ5YDM4CJ46ZRQ, seg_pos={0 0}, upload=189623]: Long tail cancellation due to optimal threshold
Mar 18 21:37:22.456701058 (197h26m25.472370669s) [noise=true , address=145.79.227.37:4566 , piece_id=CDLXWWDZWSXAUILEWL34GDA6ZNMYDYCETMQ5DDA474H4K4JGA6NQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F): protocol: expected piece hash; context canceled; EOF
Mar 18 21:37:22.456714788 (197h26m25.472384393s) [noise=true , address=145.79.227.37:4566 , piece_id=CDLXWWDZWSXAUILEWL34GDA6ZNMYDYCETMQ5DDA474H4K4JGA6NQ, seg_pos={0 0}, upload=189623]: Long tail cancellation due to optimal threshold
Mar 18 21:37:22.457052179 (197h26m25.47272179s) [noise=true , address=77.90.31.19:4845 , piece_id=53YIOMVMGMQYUKUQBWWDWLUI22FSBQEKBIGZTCPADBO2XRONWTGA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ysgMdZQcnmzQ5Vfv3RadZN8widTZKrqDnDSthxkYEUXKgRxxF): protocol: expected piece hash; context canceled; EOF
Mar 18 21:37:22.457069637 (197h26m25.472739249s) [noise=true , address=77.90.31.19:4845 , piece_id=53YIOMVMGMQYUKUQBWWDWLUI22FSBQEKBIGZTCPADBO2XRONWTGA, seg_pos={0 0}, upload=189623]: Long tail cancellation due to optimal threshold
Mar 18 21:37:22.457128468 (197h26m25.472798071s) [seg_pos={0 0}, upload=189623]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Mar 18 21:37:22.457274145 (197h26m25.472943746s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 18 21:37:22.457281197 (197h26m25.472950797s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 18 21:37:38.941520305 (197h26m41.957189918s) [upload=189623]: Done uploading segments...
Mar 18 21:37:38.941544429 (197h26m41.957214028s) [upload=189623]: Upload finished. err: <nil>
Duration: 2.497710011s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
3.906348623s trace:
https://jaeger.dev.storj.io/trace/2e6fa05e97e090
FAIL (2026-03-18 21:38:29 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 3.906460303s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................................................XX..] duration:
2.447896471s trace:
https://jaeger.dev.storj.io/trace/1fea1be3553256d8
FAIL (2026-03-18 21:38:14 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 18 21:37:13.407226914 (197h26m16.422896563s) [upload=189623]: Starting upload
Mar 18 21:37:13.407304870 (197h26m16.422974473s) [upload=189623]: Done starting upload
Mar 18 21:37:13.407540972 (197h26m16.423210564s) [upload=189623]: Uploading segments...
Mar 18 21:37:13.408026524 (197h26m16.423696147s) [upload=189623]: Got next segment. Inline: false
Mar 18 21:37:13.408123553 (197h26m16.423793153s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 18 21:37:13.408130704 (197h26m16.423800304s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 18 21:37:21.655023605 (197h26m24.670693205s) [seg_pos={0 0}, upload=189623]: Begin upload segment...
Mar 18 21:37:21.655094469 (197h26m24.670764059s) [seg_pos={0 0}, upload=189623]: Done begin upload segment.
Mar 18 21:37:21.655129337 (197h26m24.67079894s) [noise=true , address=145.79.227.4:4189 , piece_id=NMJ2X5ZX7D3U7KRZZYC2MC4XAAJNPQAZAWHLLBY2HNXSZCKCQCUQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.656236260 (197h26m24.671905872s) [noise=true , address=145.79.227.37:4566 , piece_id=CDLXWWDZWSXAUILEWL34GDA6ZNMYDYCETMQ5DDA474H4K4JGA6NQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.656865510 (197h26m24.672535113s) [noise=true , address=145.79.227.20:4439 , piece_id=3FWVMDZ7DLJV652FESYLUQS6CRTDL4ESGMHWEATRSFTBY2B6LTLA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.657306588 (197h26m24.672976189s) [noise=true , address=145.79.227.35:4486 , piece_id=RSLN2AZ2HSBPPU7XTUASMYU2Y2PFZN4YW6ZMPIOJNZTEWKWEQXJA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.657690602 (197h26m24.673360202s) [noise=true , address=77.90.31.19:4845 , piece_id=53YIOMVMGMQYUKUQBWWDWLUI22FSBQEKBIGZTCPADBO2XRONWTGA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.658444416 (197h26m24.674114022s) [noise=true , address=145.79.227.8:4335 , piece_id=U3ICUMTLPLHULI57IQ3V3H4VVEEAXSMQ4GKVFXZOUYWRYN6O2TYA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.659018578 (197h26m24.674688178s) [noise=true , address=145.79.227.15:4877 , piece_id=BQJN5RVMLYB7IIX3IFHYVFN3D2A5GYPV5L3Q6FWJ3VKVWH4NZWRA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.659604237 (197h26m24.675273838s) [noise=true , address=145.79.227.26:4657 , piece_id=O6UZJ7OLIA4BJQ6IRCI5ERXO3IURKV3GLJVG476KSPMRVQGY7LYA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.660277318 (197h26m24.675946921s) [noise=true , address=145.79.227.46:4601 , piece_id=3SWEFW5XZ3DZLPLKJZE3NBRRJCDZ7F7TDA5UXM7QBJUOWZTVPUPQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.660830355 (197h26m24.67649996s) [noise=true , address=45.91.116.7:4012 , piece_id=2JJXJQZO47ZTD7PFKABUENRESMEJMJNKAYCV6E5PKPAPO3IOKNOA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.661333927 (197h26m24.677003527s) [noise=true , address=145.79.227.3:4149 , piece_id=O4PG4M6HCKSRJZMO6KHNSFCRF5JRD2RWLNOMQGPGN2BS6TN2B2RQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.664022598 (197h26m24.679692201s) [noise=true , address=145.79.227.18:4011 , piece_id=NLH7AVODIX6ICBEYYN5JXGGLWLVOXZVYA3XWEOGUEQCW4HZTHEFQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.666068628 (197h26m24.681738234s) [noise=true , address=145.79.227.48:4306 , piece_id=FOUGOBCQYYPKA45WADXNNU4CRYNJDWVFVIFBF6VRKINXKAVXNBTQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.667030758 (197h26m24.682700369s) [noise=true , address=77.90.31.14:4663 , piece_id=IWCMYCJXN32SUB2CBFCV5TVGBL6WBISPBAZCLONZ5YDM4CJ46ZRQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.671055812 (197h26m24.686725418s) [noise=true , address=145.79.227.39:4018 , piece_id=DBAKT7XQX5IZVLEXDY63HP4NXGZVF6RTFGTNSCYKNX6EFAWVJTKQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.677611108 (197h26m24.693280721s) [noise=true , address=145.79.227.33:4874 , piece_id=NLQ737OMRU53WHBROA6TWFT4JU2RWL6AFGAVFFNWLHHEHMHFU5LQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:22.277424525 (197h26m25.293094127s) [upload=189623]: Next returned nil segment
Mar 18 21:37:22.277443392 (197h26m25.293113004s) [upload=189623]: Waiting for error group managing segments...
Mar 18 21:37:22.414377107 (197h26m25.430046707s) [noise=true , address=145.79.227.48:4306 , piece_id=FOUGOBCQYYPKA45WADXNNU4CRYNJDWVFVIFBF6VRKINXKAVXNBTQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.418771029 (197h26m25.434440635s) [noise=true , address=145.79.227.39:4018 , piece_id=DBAKT7XQX5IZVLEXDY63HP4NXGZVF6RTFGTNSCYKNX6EFAWVJTKQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.420950984 (197h26m25.436620574s) [noise=true , address=145.79.227.8:4335 , piece_id=U3ICUMTLPLHULI57IQ3V3H4VVEEAXSMQ4GKVFXZOUYWRYN6O2TYA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.421003817 (197h26m25.436673414s) [noise=true , address=145.79.227.4:4189 , piece_id=NMJ2X5ZX7D3U7KRZZYC2MC4XAAJNPQAZAWHLLBY2HNXSZCKCQCUQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.421987949 (197h26m25.437657549s) [noise=true , address=145.79.227.18:4011 , piece_id=NLH7AVODIX6ICBEYYN5JXGGLWLVOXZVYA3XWEOGUEQCW4HZTHEFQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.422142469 (197h26m25.437812071s) [noise=true , address=145.79.227.26:4657 , piece_id=O6UZJ7OLIA4BJQ6IRCI5ERXO3IURKV3GLJVG476KSPMRVQGY7LYA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.425641685 (197h26m25.441311285s) [noise=true , address=145.79.227.20:4439 , piece_id=3FWVMDZ7DLJV652FESYLUQS6CRTDL4ESGMHWEATRSFTBY2B6LTLA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.426101126 (197h26m25.44177072s) [noise=true , address=145.79.227.3:4149 , piece_id=O4PG4M6HCKSRJZMO6KHNSFCRF5JRD2RWLNOMQGPGN2BS6TN2B2RQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.427039270 (197h26m25.44270887s) [noise=true , address=145.79.227.15:4877 , piece_id=BQJN5RVMLYB7IIX3IFHYVFN3D2A5GYPV5L3Q6FWJ3VKVWH4NZWRA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.427616837 (197h26m25.443286441s) [noise=true , address=145.79.227.46:4601 , piece_id=3SWEFW5XZ3DZLPLKJZE3NBRRJCDZ7F7TDA5UXM7QBJUOWZTVPUPQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.431130437 (197h26m25.446800038s) [noise=true , address=45.91.116.7:4012 , piece_id=2JJXJQZO47ZTD7PFKABUENRESMEJMJNKAYCV6E5PKPAPO3IOKNOA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.431183849 (197h26m25.446853442s) [noise=true , address=145.79.227.33:4874 , piece_id=NLQ737OMRU53WHBROA6TWFT4JU2RWL6AFGAVFFNWLHHEHMHFU5LQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.456250331 (197h26m25.471919937s) [noise=true , address=145.79.227.35:4486 , piece_id=RSLN2AZ2HSBPPU7XTUASMYU2Y2PFZN4YW6ZMPIOJNZTEWKWEQXJA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.456277546 (197h26m25.47194715s) [seg_pos={0 0}, upload=189623]: Satisfied cohort requirements after 13 pieces
Mar 18 21:37:22.456632071 (197h26m25.472301676s) [noise=true , address=77.90.31.14:4663 , piece_id=IWCMYCJXN32SUB2CBFCV5TVGBL6WBISPBAZCLONZ5YDM4CJ46ZRQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Mar 18 21:37:22.456649964 (197h26m25.47231957s) [noise=true , address=77.90.31.14:4663 , piece_id=IWCMYCJXN32SUB2CBFCV5TVGBL6WBISPBAZCLONZ5YDM4CJ46ZRQ, seg_pos={0 0}, upload=189623]: Long tail cancellation due to optimal threshold
Mar 18 21:37:22.456701058 (197h26m25.472370669s) [noise=true , address=145.79.227.37:4566 , piece_id=CDLXWWDZWSXAUILEWL34GDA6ZNMYDYCETMQ5DDA474H4K4JGA6NQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F): protocol: expected piece hash; context canceled; EOF
Mar 18 21:37:22.456714788 (197h26m25.472384393s) [noise=true , address=145.79.227.37:4566 , piece_id=CDLXWWDZWSXAUILEWL34GDA6ZNMYDYCETMQ5DDA474H4K4JGA6NQ, seg_pos={0 0}, upload=189623]: Long tail cancellation due to optimal threshold
Mar 18 21:37:22.457052179 (197h26m25.47272179s) [noise=true , address=77.90.31.19:4845 , piece_id=53YIOMVMGMQYUKUQBWWDWLUI22FSBQEKBIGZTCPADBO2XRONWTGA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ysgMdZQcnmzQ5Vfv3RadZN8widTZKrqDnDSthxkYEUXKgRxxF): protocol: expected piece hash; context canceled; EOF
Mar 18 21:37:22.457069637 (197h26m25.472739249s) [noise=true , address=77.90.31.19:4845 , piece_id=53YIOMVMGMQYUKUQBWWDWLUI22FSBQEKBIGZTCPADBO2XRONWTGA, seg_pos={0 0}, upload=189623]: Long tail cancellation due to optimal threshold
Mar 18 21:37:22.457128468 (197h26m25.472798071s) [seg_pos={0 0}, upload=189623]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Mar 18 21:37:22.457274145 (197h26m25.472943746s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 18 21:37:22.457281197 (197h26m25.472950797s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 18 21:37:38.941520305 (197h26m41.957189918s) [upload=189623]: Done uploading segments...
Mar 18 21:37:38.941544429 (197h26m41.957214028s) [upload=189623]: Upload finished. err: <nil>
Duration: 2.447938709s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................................................XX.] duration:
20.381875867s trace:
https://jaeger.dev.storj.io/trace/1fea1be3553256d8
FAIL (2026-03-18 21:38:07 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 18 21:37:13.407226914 (197h26m16.422896563s) [upload=189623]: Starting upload
Mar 18 21:37:13.407304870 (197h26m16.422974473s) [upload=189623]: Done starting upload
Mar 18 21:37:13.407540972 (197h26m16.423210564s) [upload=189623]: Uploading segments...
Mar 18 21:37:13.408026524 (197h26m16.423696147s) [upload=189623]: Got next segment. Inline: false
Mar 18 21:37:13.408123553 (197h26m16.423793153s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 18 21:37:13.408130704 (197h26m16.423800304s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 18 21:37:21.655023605 (197h26m24.670693205s) [seg_pos={0 0}, upload=189623]: Begin upload segment...
Mar 18 21:37:21.655094469 (197h26m24.670764059s) [seg_pos={0 0}, upload=189623]: Done begin upload segment.
Mar 18 21:37:21.655129337 (197h26m24.67079894s) [noise=true , address=145.79.227.4:4189 , piece_id=NMJ2X5ZX7D3U7KRZZYC2MC4XAAJNPQAZAWHLLBY2HNXSZCKCQCUQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.656236260 (197h26m24.671905872s) [noise=true , address=145.79.227.37:4566 , piece_id=CDLXWWDZWSXAUILEWL34GDA6ZNMYDYCETMQ5DDA474H4K4JGA6NQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.656865510 (197h26m24.672535113s) [noise=true , address=145.79.227.20:4439 , piece_id=3FWVMDZ7DLJV652FESYLUQS6CRTDL4ESGMHWEATRSFTBY2B6LTLA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.657306588 (197h26m24.672976189s) [noise=true , address=145.79.227.35:4486 , piece_id=RSLN2AZ2HSBPPU7XTUASMYU2Y2PFZN4YW6ZMPIOJNZTEWKWEQXJA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.657690602 (197h26m24.673360202s) [noise=true , address=77.90.31.19:4845 , piece_id=53YIOMVMGMQYUKUQBWWDWLUI22FSBQEKBIGZTCPADBO2XRONWTGA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.658444416 (197h26m24.674114022s) [noise=true , address=145.79.227.8:4335 , piece_id=U3ICUMTLPLHULI57IQ3V3H4VVEEAXSMQ4GKVFXZOUYWRYN6O2TYA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.659018578 (197h26m24.674688178s) [noise=true , address=145.79.227.15:4877 , piece_id=BQJN5RVMLYB7IIX3IFHYVFN3D2A5GYPV5L3Q6FWJ3VKVWH4NZWRA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.659604237 (197h26m24.675273838s) [noise=true , address=145.79.227.26:4657 , piece_id=O6UZJ7OLIA4BJQ6IRCI5ERXO3IURKV3GLJVG476KSPMRVQGY7LYA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.660277318 (197h26m24.675946921s) [noise=true , address=145.79.227.46:4601 , piece_id=3SWEFW5XZ3DZLPLKJZE3NBRRJCDZ7F7TDA5UXM7QBJUOWZTVPUPQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.660830355 (197h26m24.67649996s) [noise=true , address=45.91.116.7:4012 , piece_id=2JJXJQZO47ZTD7PFKABUENRESMEJMJNKAYCV6E5PKPAPO3IOKNOA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.661333927 (197h26m24.677003527s) [noise=true , address=145.79.227.3:4149 , piece_id=O4PG4M6HCKSRJZMO6KHNSFCRF5JRD2RWLNOMQGPGN2BS6TN2B2RQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.664022598 (197h26m24.679692201s) [noise=true , address=145.79.227.18:4011 , piece_id=NLH7AVODIX6ICBEYYN5JXGGLWLVOXZVYA3XWEOGUEQCW4HZTHEFQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.666068628 (197h26m24.681738234s) [noise=true , address=145.79.227.48:4306 , piece_id=FOUGOBCQYYPKA45WADXNNU4CRYNJDWVFVIFBF6VRKINXKAVXNBTQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.667030758 (197h26m24.682700369s) [noise=true , address=77.90.31.14:4663 , piece_id=IWCMYCJXN32SUB2CBFCV5TVGBL6WBISPBAZCLONZ5YDM4CJ46ZRQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.671055812 (197h26m24.686725418s) [noise=true , address=145.79.227.39:4018 , piece_id=DBAKT7XQX5IZVLEXDY63HP4NXGZVF6RTFGTNSCYKNX6EFAWVJTKQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.677611108 (197h26m24.693280721s) [noise=true , address=145.79.227.33:4874 , piece_id=NLQ737OMRU53WHBROA6TWFT4JU2RWL6AFGAVFFNWLHHEHMHFU5LQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:22.277424525 (197h26m25.293094127s) [upload=189623]: Next returned nil segment
Mar 18 21:37:22.277443392 (197h26m25.293113004s) [upload=189623]: Waiting for error group managing segments...
Mar 18 21:37:22.414377107 (197h26m25.430046707s) [noise=true , address=145.79.227.48:4306 , piece_id=FOUGOBCQYYPKA45WADXNNU4CRYNJDWVFVIFBF6VRKINXKAVXNBTQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.418771029 (197h26m25.434440635s) [noise=true , address=145.79.227.39:4018 , piece_id=DBAKT7XQX5IZVLEXDY63HP4NXGZVF6RTFGTNSCYKNX6EFAWVJTKQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.420950984 (197h26m25.436620574s) [noise=true , address=145.79.227.8:4335 , piece_id=U3ICUMTLPLHULI57IQ3V3H4VVEEAXSMQ4GKVFXZOUYWRYN6O2TYA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.421003817 (197h26m25.436673414s) [noise=true , address=145.79.227.4:4189 , piece_id=NMJ2X5ZX7D3U7KRZZYC2MC4XAAJNPQAZAWHLLBY2HNXSZCKCQCUQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.421987949 (197h26m25.437657549s) [noise=true , address=145.79.227.18:4011 , piece_id=NLH7AVODIX6ICBEYYN5JXGGLWLVOXZVYA3XWEOGUEQCW4HZTHEFQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.422142469 (197h26m25.437812071s) [noise=true , address=145.79.227.26:4657 , piece_id=O6UZJ7OLIA4BJQ6IRCI5ERXO3IURKV3GLJVG476KSPMRVQGY7LYA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.425641685 (197h26m25.441311285s) [noise=true , address=145.79.227.20:4439 , piece_id=3FWVMDZ7DLJV652FESYLUQS6CRTDL4ESGMHWEATRSFTBY2B6LTLA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.426101126 (197h26m25.44177072s) [noise=true , address=145.79.227.3:4149 , piece_id=O4PG4M6HCKSRJZMO6KHNSFCRF5JRD2RWLNOMQGPGN2BS6TN2B2RQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.427039270 (197h26m25.44270887s) [noise=true , address=145.79.227.15:4877 , piece_id=BQJN5RVMLYB7IIX3IFHYVFN3D2A5GYPV5L3Q6FWJ3VKVWH4NZWRA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.427616837 (197h26m25.443286441s) [noise=true , address=145.79.227.46:4601 , piece_id=3SWEFW5XZ3DZLPLKJZE3NBRRJCDZ7F7TDA5UXM7QBJUOWZTVPUPQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.431130437 (197h26m25.446800038s) [noise=true , address=45.91.116.7:4012 , piece_id=2JJXJQZO47ZTD7PFKABUENRESMEJMJNKAYCV6E5PKPAPO3IOKNOA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.431183849 (197h26m25.446853442s) [noise=true , address=145.79.227.33:4874 , piece_id=NLQ737OMRU53WHBROA6TWFT4JU2RWL6AFGAVFFNWLHHEHMHFU5LQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.456250331 (197h26m25.471919937s) [noise=true , address=145.79.227.35:4486 , piece_id=RSLN2AZ2HSBPPU7XTUASMYU2Y2PFZN4YW6ZMPIOJNZTEWKWEQXJA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.456277546 (197h26m25.47194715s) [seg_pos={0 0}, upload=189623]: Satisfied cohort requirements after 13 pieces
Mar 18 21:37:22.456632071 (197h26m25.472301676s) [noise=true , address=77.90.31.14:4663 , piece_id=IWCMYCJXN32SUB2CBFCV5TVGBL6WBISPBAZCLONZ5YDM4CJ46ZRQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Mar 18 21:37:22.456649964 (197h26m25.47231957s) [noise=true , address=77.90.31.14:4663 , piece_id=IWCMYCJXN32SUB2CBFCV5TVGBL6WBISPBAZCLONZ5YDM4CJ46ZRQ, seg_pos={0 0}, upload=189623]: Long tail cancellation due to optimal threshold
Mar 18 21:37:22.456701058 (197h26m25.472370669s) [noise=true , address=145.79.227.37:4566 , piece_id=CDLXWWDZWSXAUILEWL34GDA6ZNMYDYCETMQ5DDA474H4K4JGA6NQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F): protocol: expected piece hash; context canceled; EOF
Mar 18 21:37:22.456714788 (197h26m25.472384393s) [noise=true , address=145.79.227.37:4566 , piece_id=CDLXWWDZWSXAUILEWL34GDA6ZNMYDYCETMQ5DDA474H4K4JGA6NQ, seg_pos={0 0}, upload=189623]: Long tail cancellation due to optimal threshold
Mar 18 21:37:22.457052179 (197h26m25.47272179s) [noise=true , address=77.90.31.19:4845 , piece_id=53YIOMVMGMQYUKUQBWWDWLUI22FSBQEKBIGZTCPADBO2XRONWTGA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ysgMdZQcnmzQ5Vfv3RadZN8widTZKrqDnDSthxkYEUXKgRxxF): protocol: expected piece hash; context canceled; EOF
Mar 18 21:37:22.457069637 (197h26m25.472739249s) [noise=true , address=77.90.31.19:4845 , piece_id=53YIOMVMGMQYUKUQBWWDWLUI22FSBQEKBIGZTCPADBO2XRONWTGA, seg_pos={0 0}, upload=189623]: Long tail cancellation due to optimal threshold
Mar 18 21:37:22.457128468 (197h26m25.472798071s) [seg_pos={0 0}, upload=189623]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Mar 18 21:37:22.457274145 (197h26m25.472943746s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 18 21:37:22.457281197 (197h26m25.472950797s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 18 21:37:38.941520305 (197h26m41.957189918s) [upload=189623]: Done uploading segments...
Mar 18 21:37:38.941544429 (197h26m41.957214028s) [upload=189623]: Upload finished. err: <nil>
Duration: 20.381903775s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
29.459899286s trace:
https://jaeger.dev.storj.io/trace/2e6fa05e97e090
FAIL (2026-03-18 21:38:07 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 29.4599173s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [........................................................XX..] duration:
17.551497613s trace:
https://jaeger.dev.storj.io/trace/94bb3ca45ba55fa
FAIL (2026-03-18 21:38:07 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 17.551539569s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [.........................................................XX.] duration:
29.716997592s trace:
https://jaeger.dev.storj.io/trace/94bb3ca45ba55fa
FAIL (2026-03-18 21:37:48 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 29.717020809s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
30.003252209s trace:
https://jaeger.dev.storj.io/trace/1fea1be3553256d8
FAIL (2026-03-18 21:37:43 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 18 21:37:13.407226914 (197h26m16.422896563s) [upload=189623]: Starting upload
Mar 18 21:37:13.407304870 (197h26m16.422974473s) [upload=189623]: Done starting upload
Mar 18 21:37:13.407540972 (197h26m16.423210564s) [upload=189623]: Uploading segments...
Mar 18 21:37:13.408026524 (197h26m16.423696147s) [upload=189623]: Got next segment. Inline: false
Mar 18 21:37:13.408123553 (197h26m16.423793153s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 18 21:37:13.408130704 (197h26m16.423800304s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 18 21:37:21.655023605 (197h26m24.670693205s) [seg_pos={0 0}, upload=189623]: Begin upload segment...
Mar 18 21:37:21.655094469 (197h26m24.670764059s) [seg_pos={0 0}, upload=189623]: Done begin upload segment.
Mar 18 21:37:21.655129337 (197h26m24.67079894s) [noise=true , address=145.79.227.4:4189 , piece_id=NMJ2X5ZX7D3U7KRZZYC2MC4XAAJNPQAZAWHLLBY2HNXSZCKCQCUQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.656236260 (197h26m24.671905872s) [noise=true , address=145.79.227.37:4566 , piece_id=CDLXWWDZWSXAUILEWL34GDA6ZNMYDYCETMQ5DDA474H4K4JGA6NQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.656865510 (197h26m24.672535113s) [noise=true , address=145.79.227.20:4439 , piece_id=3FWVMDZ7DLJV652FESYLUQS6CRTDL4ESGMHWEATRSFTBY2B6LTLA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.657306588 (197h26m24.672976189s) [noise=true , address=145.79.227.35:4486 , piece_id=RSLN2AZ2HSBPPU7XTUASMYU2Y2PFZN4YW6ZMPIOJNZTEWKWEQXJA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.657690602 (197h26m24.673360202s) [noise=true , address=77.90.31.19:4845 , piece_id=53YIOMVMGMQYUKUQBWWDWLUI22FSBQEKBIGZTCPADBO2XRONWTGA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.658444416 (197h26m24.674114022s) [noise=true , address=145.79.227.8:4335 , piece_id=U3ICUMTLPLHULI57IQ3V3H4VVEEAXSMQ4GKVFXZOUYWRYN6O2TYA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.659018578 (197h26m24.674688178s) [noise=true , address=145.79.227.15:4877 , piece_id=BQJN5RVMLYB7IIX3IFHYVFN3D2A5GYPV5L3Q6FWJ3VKVWH4NZWRA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.659604237 (197h26m24.675273838s) [noise=true , address=145.79.227.26:4657 , piece_id=O6UZJ7OLIA4BJQ6IRCI5ERXO3IURKV3GLJVG476KSPMRVQGY7LYA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.660277318 (197h26m24.675946921s) [noise=true , address=145.79.227.46:4601 , piece_id=3SWEFW5XZ3DZLPLKJZE3NBRRJCDZ7F7TDA5UXM7QBJUOWZTVPUPQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.660830355 (197h26m24.67649996s) [noise=true , address=45.91.116.7:4012 , piece_id=2JJXJQZO47ZTD7PFKABUENRESMEJMJNKAYCV6E5PKPAPO3IOKNOA, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.661333927 (197h26m24.677003527s) [noise=true , address=145.79.227.3:4149 , piece_id=O4PG4M6HCKSRJZMO6KHNSFCRF5JRD2RWLNOMQGPGN2BS6TN2B2RQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.664022598 (197h26m24.679692201s) [noise=true , address=145.79.227.18:4011 , piece_id=NLH7AVODIX6ICBEYYN5JXGGLWLVOXZVYA3XWEOGUEQCW4HZTHEFQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.666068628 (197h26m24.681738234s) [noise=true , address=145.79.227.48:4306 , piece_id=FOUGOBCQYYPKA45WADXNNU4CRYNJDWVFVIFBF6VRKINXKAVXNBTQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.667030758 (197h26m24.682700369s) [noise=true , address=77.90.31.14:4663 , piece_id=IWCMYCJXN32SUB2CBFCV5TVGBL6WBISPBAZCLONZ5YDM4CJ46ZRQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.671055812 (197h26m24.686725418s) [noise=true , address=145.79.227.39:4018 , piece_id=DBAKT7XQX5IZVLEXDY63HP4NXGZVF6RTFGTNSCYKNX6EFAWVJTKQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:21.677611108 (197h26m24.693280721s) [noise=true , address=145.79.227.33:4874 , piece_id=NLQ737OMRU53WHBROA6TWFT4JU2RWL6AFGAVFFNWLHHEHMHFU5LQ, seg_pos={0 0}, upload=189623]: Uploading piece...
Mar 18 21:37:22.277424525 (197h26m25.293094127s) [upload=189623]: Next returned nil segment
Mar 18 21:37:22.277443392 (197h26m25.293113004s) [upload=189623]: Waiting for error group managing segments...
Mar 18 21:37:22.414377107 (197h26m25.430046707s) [noise=true , address=145.79.227.48:4306 , piece_id=FOUGOBCQYYPKA45WADXNNU4CRYNJDWVFVIFBF6VRKINXKAVXNBTQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.418771029 (197h26m25.434440635s) [noise=true , address=145.79.227.39:4018 , piece_id=DBAKT7XQX5IZVLEXDY63HP4NXGZVF6RTFGTNSCYKNX6EFAWVJTKQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.420950984 (197h26m25.436620574s) [noise=true , address=145.79.227.8:4335 , piece_id=U3ICUMTLPLHULI57IQ3V3H4VVEEAXSMQ4GKVFXZOUYWRYN6O2TYA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.421003817 (197h26m25.436673414s) [noise=true , address=145.79.227.4:4189 , piece_id=NMJ2X5ZX7D3U7KRZZYC2MC4XAAJNPQAZAWHLLBY2HNXSZCKCQCUQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.421987949 (197h26m25.437657549s) [noise=true , address=145.79.227.18:4011 , piece_id=NLH7AVODIX6ICBEYYN5JXGGLWLVOXZVYA3XWEOGUEQCW4HZTHEFQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.422142469 (197h26m25.437812071s) [noise=true , address=145.79.227.26:4657 , piece_id=O6UZJ7OLIA4BJQ6IRCI5ERXO3IURKV3GLJVG476KSPMRVQGY7LYA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.425641685 (197h26m25.441311285s) [noise=true , address=145.79.227.20:4439 , piece_id=3FWVMDZ7DLJV652FESYLUQS6CRTDL4ESGMHWEATRSFTBY2B6LTLA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.426101126 (197h26m25.44177072s) [noise=true , address=145.79.227.3:4149 , piece_id=O4PG4M6HCKSRJZMO6KHNSFCRF5JRD2RWLNOMQGPGN2BS6TN2B2RQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.427039270 (197h26m25.44270887s) [noise=true , address=145.79.227.15:4877 , piece_id=BQJN5RVMLYB7IIX3IFHYVFN3D2A5GYPV5L3Q6FWJ3VKVWH4NZWRA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.427616837 (197h26m25.443286441s) [noise=true , address=145.79.227.46:4601 , piece_id=3SWEFW5XZ3DZLPLKJZE3NBRRJCDZ7F7TDA5UXM7QBJUOWZTVPUPQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.431130437 (197h26m25.446800038s) [noise=true , address=45.91.116.7:4012 , piece_id=2JJXJQZO47ZTD7PFKABUENRESMEJMJNKAYCV6E5PKPAPO3IOKNOA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.431183849 (197h26m25.446853442s) [noise=true , address=145.79.227.33:4874 , piece_id=NLQ737OMRU53WHBROA6TWFT4JU2RWL6AFGAVFFNWLHHEHMHFU5LQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.456250331 (197h26m25.471919937s) [noise=true , address=145.79.227.35:4486 , piece_id=RSLN2AZ2HSBPPU7XTUASMYU2Y2PFZN4YW6ZMPIOJNZTEWKWEQXJA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: <nil>
Mar 18 21:37:22.456277546 (197h26m25.47194715s) [seg_pos={0 0}, upload=189623]: Satisfied cohort requirements after 13 pieces
Mar 18 21:37:22.456632071 (197h26m25.472301676s) [noise=true , address=77.90.31.14:4663 , piece_id=IWCMYCJXN32SUB2CBFCV5TVGBL6WBISPBAZCLONZ5YDM4CJ46ZRQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Mar 18 21:37:22.456649964 (197h26m25.47231957s) [noise=true , address=77.90.31.14:4663 , piece_id=IWCMYCJXN32SUB2CBFCV5TVGBL6WBISPBAZCLONZ5YDM4CJ46ZRQ, seg_pos={0 0}, upload=189623]: Long tail cancellation due to optimal threshold
Mar 18 21:37:22.456701058 (197h26m25.472370669s) [noise=true , address=145.79.227.37:4566 , piece_id=CDLXWWDZWSXAUILEWL34GDA6ZNMYDYCETMQ5DDA474H4K4JGA6NQ, seg_pos={0 0}, upload=189623]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F): protocol: expected piece hash; context canceled; EOF
Mar 18 21:37:22.456714788 (197h26m25.472384393s) [noise=true , address=145.79.227.37:4566 , piece_id=CDLXWWDZWSXAUILEWL34GDA6ZNMYDYCETMQ5DDA474H4K4JGA6NQ, seg_pos={0 0}, upload=189623]: Long tail cancellation due to optimal threshold
Mar 18 21:37:22.457052179 (197h26m25.47272179s) [noise=true , address=77.90.31.19:4845 , piece_id=53YIOMVMGMQYUKUQBWWDWLUI22FSBQEKBIGZTCPADBO2XRONWTGA, seg_pos={0 0}, upload=189623]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ysgMdZQcnmzQ5Vfv3RadZN8widTZKrqDnDSthxkYEUXKgRxxF): protocol: expected piece hash; context canceled; EOF
Mar 18 21:37:22.457069637 (197h26m25.472739249s) [noise=true , address=77.90.31.19:4845 , piece_id=53YIOMVMGMQYUKUQBWWDWLUI22FSBQEKBIGZTCPADBO2XRONWTGA, seg_pos={0 0}, upload=189623]: Long tail cancellation due to optimal threshold
Mar 18 21:37:22.457128468 (197h26m25.472798071s) [seg_pos={0 0}, upload=189623]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Mar 18 21:37:22.457274145 (197h26m25.472943746s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 18 21:37:22.457281197 (197h26m25.472950797s) [upload=189623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 18 21:37:38.941520305 (197h26m41.957189918s) [upload=189623]: Done uploading segments...
Mar 18 21:37:38.941544429 (197h26m41.957214028s) [upload=189623]: Upload finished. err: <nil>
Duration: 30.003300831s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [X.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXXXXXXXXXXXXX] duration:
5.001423339s trace:
https://jaeger.dev.storj.io/trace/468bbab930a6aba3
FAIL (2026-03-18 21:37:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001446806s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.005848934s trace:
https://jaeger.dev.storj.io/trace/2e6fa05e97e090
FAIL (2026-03-18 21:37:37 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 30.00587904s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
30.001565979s trace:
https://jaeger.dev.storj.io/trace/94bb3ca45ba55fa
FAIL (2026-03-18 21:37:18 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 30.001591081s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.00198765s trace:
https://jaeger.dev.storj.io/trace/5cc28c3f4ec2c38d
FAIL (2026-03-18 21:36:55 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 18 21:36:25.661178637 (197h25m28.676848241s) [upload=189613]: Starting upload
Mar 18 21:36:25.661255531 (197h25m28.676925137s) [upload=189613]: Done starting upload
Mar 18 21:36:25.661445021 (197h25m28.677114689s) [upload=189613]: Uploading segments...
Mar 18 21:36:25.661757363 (197h25m28.677426953s) [upload=189613]: Got next segment. Inline: false
Mar 18 21:36:25.661811713 (197h25m28.677481313s) [upload=189613]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 18 21:36:25.661817155 (197h25m28.677486755s) [upload=189613]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 18 21:36:29.871559002 (197h25m32.887228705s) [seg_pos={0 0}, upload=189613]: Begin upload segment...
Mar 18 21:36:29.871630531 (197h25m32.88730014s) [seg_pos={0 0}, upload=189613]: Done begin upload segment.
Mar 18 21:36:29.871666194 (197h25m32.887335808s) [noise=true , address=45.91.116.5:4297 , piece_id=GAU5LNOZRGGKCU5DH5NGU6LQKMFMLASDVISVJMBG3G35HL33NMJQ, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.871693296 (197h25m32.887362907s) [noise=true , address=145.79.227.20:4443 , piece_id=65TZ3YHW2QHJMRHHBLO3XM3TYEDZWXC5VH77RNX3PVYM6RSEJZSA, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.872222878 (197h25m32.887892486s) [noise=true , address=77.90.31.16:4736 , piece_id=TSMFXRIZGXB5FQOQJFAZDUIGO3NZPFBPGXVOE7D6XFOJZYDGE4LA, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.872714975 (197h25m32.888384629s) [noise=true , address=145.79.227.32:4833 , piece_id=XLRHUG77ALLXFOSNPNMNABC7ZC5JHAFFVCMMHRZSWVKHKF6G7L6A, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.873182998 (197h25m32.888852608s) [noise=true , address=145.79.227.34:4907 , piece_id=RSBGXG3U4NPXAHV6IMYNPNTJVKZHMR4MYTZAUW2SREHUPOLT2UHQ, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.873652126 (197h25m32.889321736s) [noise=true , address=45.91.116.4:4937 , piece_id=7Y4U5CJKIWEYRPIGNSLVLCXS6XFVNXVJW3EIRGXJ3SH73KSK6BCA, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.874266095 (197h25m32.889935705s) [noise=true , address=145.79.227.47:4265 , piece_id=CUQ4N7VKRTF27Y6YHHK6NIWYIBYYPT52WL6GWEA5L4Y4ZMO5YUWQ, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.874849824 (197h25m32.890519478s) [noise=true , address=77.90.31.21:4880 , piece_id=6ADT3RZ23JWYNBJ7FNN5AXXTJP32ZILTA4ZQH4SZS6DZWNU4GLRA, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.875602075 (197h25m32.891271693s) [noise=true , address=145.79.227.42:4121 , piece_id=AGVG6ZBOXDK4LHQP7IU7D547EB3CI5V3UI2AC44ZHJUPTPYPDY6Q, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.877876705 (197h25m32.893546319s) [noise=true , address=145.79.227.14:4843 , piece_id=X7M5GIS5MXYKDVGTCZUORKYGWJAMB4S6EYMZTI63KBLBJKHHUDNQ, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.879769693 (197h25m32.895439306s) [noise=true , address=145.79.227.33:4871 , piece_id=KJETW6RJXOFX45WF4YEGDYCLYUVN4GBB4BU63PWNA6HB434CARYA, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.881648124 (197h25m32.897317736s) [noise=true , address=145.79.227.16:4913 , piece_id=O4CGAXFWBC535N4P7L5UG7M5YYT77SQKGAADUPUFMHRPGPQHXC4A, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.883576657 (197h25m32.899246302s) [noise=true , address=145.79.227.45:4233 , piece_id=SK4FLO7E5XV2ISE6IFVXKUTU7EHV3YMREB6YZNRQ73RHWFV4ZGHA, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.885570220 (197h25m32.901239845s) [noise=true , address=145.79.227.18:4011 , piece_id=KBUJ3DKSK4VDTN3IECHN2VHYBM7V33VI2H7I2KHBYDOYXAMFWGGA, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.887425275 (197h25m32.903094887s) [noise=true , address=45.91.116.7:4026 , piece_id=42XRE4IR3POV66S3YRBNDP4NKMSCJZY7LILZYFTISXKOFHKOOYMA, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:29.888009225 (197h25m32.903678838s) [noise=true , address=145.79.227.31:4800 , piece_id=YKRIB7MA6J2KP4D3CCL2XG63WIM7BPJJWMAO4OP2ZVKQE357A4VQ, seg_pos={0 0}, upload=189613]: Uploading piece...
Mar 18 21:36:30.503714547 (197h25m33.519384161s) [upload=189613]: Next returned nil segment
Mar 18 21:36:30.503733859 (197h25m33.519403473s) [upload=189613]: Waiting for error group managing segments...
Mar 18 21:36:30.626836203 (197h25m33.642505808s) [noise=true , address=45.91.116.4:4937 , piece_id=7Y4U5CJKIWEYRPIGNSLVLCXS6XFVNXVJW3EIRGXJ3SH73KSK6BCA, seg_pos={0 0}, upload=189613]: Done uploading piece. err: <nil>
Mar 18 21:36:30.628306218 (197h25m33.64397582s) [noise=true , address=145.79.227.34:4907 , piece_id=RSBGXG3U4NPXAHV6IMYNPNTJVKZHMR4MYTZAUW2SREHUPOLT2UHQ, seg_pos={0 0}, upload=189613]: Done uploading piece. err: <nil>
Mar 18 21:36:30.643647922 (197h25m33.659317523s) [noise=true , address=145.79.227.16:4913 , piece_id=O4CGAXFWBC535N4P7L5UG7M5YYT77SQKGAADUPUFMHRPGPQHXC4A, seg_pos={0 0}, upload=189613]: Done uploading piece. err: <nil>
Mar 18 21:36:30.644517053 (197h25m33.660186655s) [noise=true , address=145.79.227.45:4233 , piece_id=SK4FLO7E5XV2ISE6IFVXKUTU7EHV3YMREB6YZNRQ73RHWFV4ZGHA, seg_pos={0 0}, upload=189613]: Done uploading piece. err: <nil>
Mar 18 21:36:30.646400794 (197h25m33.662070397s) [noise=true , address=145.79.227.14:4843 , piece_id=X7M5GIS5MXYKDVGTCZUORKYGWJAMB4S6EYMZTI63KBLBJKHHUDNQ, seg_pos={0 0}, upload=189613]: Done uploading piece. err: <nil>
Mar 18 21:36:30.647607134 (197h25m33.663276734s) [noise=true , address=145.79.227.18:4011 , piece_id=KBUJ3DKSK4VDTN3IECHN2VHYBM7V33VI2H7I2KHBYDOYXAMFWGGA, seg_pos={0 0}, upload=189613]: Done uploading piece. err: <nil>
Mar 18 21:36:30.648701585 (197h25m33.664371186s) [noise=true , address=145.79.227.32:4833 , piece_id=XLRHUG77ALLXFOSNPNMNABC7ZC5JHAFFVCMMHRZSWVKHKF6G7L6A, seg_pos={0 0}, upload=189613]: Done uploading piece. err: <nil>
Mar 18 21:36:30.649891528 (197h25m33.66556113s) [noise=true , address=145.79.227.20:4443 , piece_id=65TZ3YHW2QHJMRHHBLO3XM3TYEDZWXC5VH77RNX3PVYM6RSEJZSA, seg_pos={0 0}, upload=189613]: Done uploading piece. err: <nil>
Mar 18 21:36:30.651098798 (197h25m33.666768403s) [noise=true , address=145.79.227.42:4121 , piece_id=AGVG6ZBOXDK4LHQP7IU7D547EB3CI5V3UI2AC44ZHJUPTPYPDY6Q, seg_pos={0 0}, upload=189613]: Done uploading piece. err: <nil>
Mar 18 21:36:30.657398228 (197h25m33.673067834s) [noise=true , address=145.79.227.31:4800 , piece_id=YKRIB7MA6J2KP4D3CCL2XG63WIM7BPJJWMAO4OP2ZVKQE357A4VQ, seg_pos={0 0}, upload=189613]: Done uploading piece. err: <nil>
Mar 18 21:36:30.658619670 (197h25m33.674289274s) [noise=true , address=145.79.227.33:4871 , piece_id=KJETW6RJXOFX45WF4YEGDYCLYUVN4GBB4BU63PWNA6HB434CARYA, seg_pos={0 0}, upload=189613]: Done uploading piece. err: <nil>
Mar 18 21:36:30.659393966 (197h25m33.675063569s) [noise=true , address=45.91.116.7:4026 , piece_id=42XRE4IR3POV66S3YRBNDP4NKMSCJZY7LILZYFTISXKOFHKOOYMA, seg_pos={0 0}, upload=189613]: Done uploading piece. err: <nil>
Mar 18 21:36:30.659990900 (197h25m33.6756605s) [noise=true , address=45.91.116.5:4297 , piece_id=GAU5LNOZRGGKCU5DH5NGU6LQKMFMLASDVISVJMBG3G35HL33NMJQ, seg_pos={0 0}, upload=189613]: Done uploading piece. err: <nil>
Mar 18 21:36:30.660003523 (197h25m33.675673126s) [seg_pos={0 0}, upload=189613]: Satisfied cohort requirements after 13 pieces
Mar 18 21:36:30.660438227 (197h25m33.676107836s) [noise=true , address=77.90.31.21:4880 , piece_id=6ADT3RZ23JWYNBJ7FNN5AXXTJP32ZILTA4ZQH4SZS6DZWNU4GLRA, seg_pos={0 0}, upload=189613]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb): protocol: expected piece hash; context canceled; EOF
Mar 18 21:36:30.660448805 (197h25m33.676118408s) [noise=true , address=77.90.31.21:4880 , piece_id=6ADT3RZ23JWYNBJ7FNN5AXXTJP32ZILTA4ZQH4SZS6DZWNU4GLRA, seg_pos={0 0}, upload=189613]: Long tail cancellation due to optimal threshold
Mar 18 21:36:30.668179972 (197h25m33.683849572s) [noise=true , address=77.90.31.16:4736 , piece_id=TSMFXRIZGXB5FQOQJFAZDUIGO3NZPFBPGXVOE7D6XFOJZYDGE4LA, seg_pos={0 0}, upload=189613]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk): protocol: expected piece hash; context canceled; EOF
Mar 18 21:36:30.668199123 (197h25m33.683868724s) [noise=true , address=77.90.31.16:4736 , piece_id=TSMFXRIZGXB5FQOQJFAZDUIGO3NZPFBPGXVOE7D6XFOJZYDGE4LA, seg_pos={0 0}, upload=189613]: Long tail cancellation due to optimal threshold
Mar 18 21:36:30.684837292 (197h25m33.700506897s) [noise=true , address=145.79.227.47:4265 , piece_id=CUQ4N7VKRTF27Y6YHHK6NIWYIBYYPT52WL6GWEA5L4Y4ZMO5YUWQ, seg_pos={0 0}, upload=189613]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ): protocol: expected piece hash; context canceled; EOF
Mar 18 21:36:30.684857338 (197h25m33.700526939s) [noise=true , address=145.79.227.47:4265 , piece_id=CUQ4N7VKRTF27Y6YHHK6NIWYIBYYPT52WL6GWEA5L4Y4ZMO5YUWQ, seg_pos={0 0}, upload=189613]: Long tail cancellation due to optimal threshold
Mar 18 21:36:30.684940607 (197h25m33.700610197s) [seg_pos={0 0}, upload=189613]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Mar 18 21:36:30.685030754 (197h25m33.700700368s) [upload=189613]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 18 21:36:30.685041091 (197h25m33.700710703s) [upload=189613]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 18 21:36:43.052045158 (197h25m46.067714768s) [upload=189613]: Done uploading segments...
Mar 18 21:36:43.052070094 (197h25m46.067739696s) [upload=189613]: Upload finished. err: <nil>
Duration: 30.00201917s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXXXXXXXXXXXX] duration:
5.016916553s trace:
https://jaeger.dev.storj.io/trace/232862972182f00b
FAIL (2026-03-18 21:36:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.016948855s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.002135961s trace:
https://jaeger.dev.storj.io/trace/210c81557c2b408d
FAIL (2026-03-18 21:36:11 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 30.002166398s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXXXXXXXXXXX] duration:
5.001552061s trace:
https://jaeger.dev.storj.io/trace/2b195108edee4a5f
FAIL (2026-03-18 21:35:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001584371s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXXXXXXXXXX] duration:
5.009018497s trace:
https://jaeger.dev.storj.io/trace/45bc8d934deed9bc
FAIL (2026-03-18 21:34:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.009048154s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXXXXXXXXX] duration:
5.001280111s trace:
https://jaeger.dev.storj.io/trace/7284b77697a9a933
FAIL (2026-03-18 21:33:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001306292s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXXXXXXXX] duration:
5.001320913s trace:
https://jaeger.dev.storj.io/trace/2194dfdd7d3fdf33
FAIL (2026-03-18 21:32:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001343221s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXXXXXXX] duration:
5.001760938s trace:
https://jaeger.dev.storj.io/trace/265e09e28f53aca4
FAIL (2026-03-18 21:32:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001794838s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXXXXXX] duration:
5.001348644s trace:
https://jaeger.dev.storj.io/trace/b20c66e096ed238
FAIL (2026-03-18 21:31:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001613199s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXXXXX] duration:
5.001522376s trace:
https://jaeger.dev.storj.io/trace/16a516a19f171be7
FAIL (2026-03-18 21:30:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001556145s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXXXX] duration:
5.001221874s trace:
https://jaeger.dev.storj.io/trace/1e5a223108da5121
FAIL (2026-03-18 21:30:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001245772s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXXX] duration:
5.001245153s trace:
https://jaeger.dev.storj.io/trace/625b24a8efc15fb6
FAIL (2026-03-18 21:28:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001269743s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XXX] duration:
5.001839613s trace:
https://jaeger.dev.storj.io/trace/221da7f6e624710c
FAIL (2026-03-18 21:27:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001872131s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......XX] duration:
5.001256559s trace:
https://jaeger.dev.storj.io/trace/2eb7cb9b848ac981
FAIL (2026-03-18 21:27:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001288879s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......X] duration:
5.001003214s trace:
https://jaeger.dev.storj.io/trace/725cf65ec8b0e3ad
FAIL (2026-03-18 21:27:05 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001027991s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
43 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX......] duration:
2.228770181s trace:
https://jaeger.dev.storj.io/trace/2088c0364ed4903d
FAIL (2026-03-18 21:26:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.228897097s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
44 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX.....] duration:
1.648688193s trace:
https://jaeger.dev.storj.io/trace/2088c0364ed4903d
FAIL (2026-03-18 21:25:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.648705703s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
45 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX....] duration:
1.819005651s trace:
https://jaeger.dev.storj.io/trace/2088c0364ed4903d
FAIL (2026-03-18 21:24:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.819036194s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
46 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX...] duration:
1.964977638s trace:
https://jaeger.dev.storj.io/trace/2088c0364ed4903d
FAIL (2026-03-18 21:23:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.965001889s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
47 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX..] duration:
1.959353884s trace:
https://jaeger.dev.storj.io/trace/2088c0364ed4903d
FAIL (2026-03-18 21:23:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.959381708s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
48 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX.] duration:
1.830455324s trace:
https://jaeger.dev.storj.io/trace/2088c0364ed4903d
FAIL (2026-03-18 21:22:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.830486778s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
49 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000746927s trace:
https://jaeger.dev.storj.io/trace/2088c0364ed4903d
FAIL (2026-03-18 21:22:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001221481s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
49 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.001797597s trace:
https://jaeger.dev.storj.io/trace/4caed5ab3798219d
FAIL (2026-03-18 21:21:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001887659s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
49 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.001950467s trace:
https://jaeger.dev.storj.io/trace/43ee508220a50230
FAIL (2026-03-18 21:21:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.00197333s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
49 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000715973s trace:
https://jaeger.dev.storj.io/trace/7465ca37562c16cc
FAIL (2026-03-18 21:20:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000740527s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
49 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXXX] duration:
5.001732819s trace:
https://jaeger.dev.storj.io/trace/4184f82551a3fc60
FAIL (2026-03-18 21:19:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001763394s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
49 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXXX] duration:
5.001079294s trace:
https://jaeger.dev.storj.io/trace/53d43427d705305c
FAIL (2026-03-18 21:18:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001110124s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
49 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXXX] duration:
5.000899545s trace:
https://jaeger.dev.storj.io/trace/15810fac6ba5f0aa
FAIL (2026-03-18 21:17:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000955767s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
49 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXXX] duration:
5.001124676s trace:
https://jaeger.dev.storj.io/trace/3633041f92b5999b
FAIL (2026-03-18 21:17:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001154111s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
49 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXXX] duration:
5.001303352s trace:
https://jaeger.dev.storj.io/trace/127205b085f42671
FAIL (2026-03-18 21:16:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001382901s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
49 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXXX] duration:
5.001012995s trace:
https://jaeger.dev.storj.io/trace/15eb47fa877e832a
FAIL (2026-03-18 21:15:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001102409s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
49 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXXX] duration:
5.001772077s trace:
https://jaeger.dev.storj.io/trace/69821541d09fc00c
FAIL (2026-03-18 21:14:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001977335s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
49 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXXX] duration:
5.000855083s trace:
https://jaeger.dev.storj.io/trace/60997998e0b26318
FAIL (2026-03-18 21:13:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000877767s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
48 out of 60 is failed (threshold=6) [.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXXX] duration:
5.000788662s trace:
https://jaeger.dev.storj.io/trace/16b070b118a9a87f
FAIL (2026-03-18 21:12:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000812454s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
47 out of 60 is failed (threshold=6) [..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX..........XXXXXXXXXXXXX] duration:
5.000859299s trace:
https://jaeger.dev.storj.io/trace/1d167bd19f51ce96
FAIL (2026-03-18 21:11:25 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000885294s
core
uplink
us-select-1
us1-placement-20
|