Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [..............X..XX.................................X.......] duration:
2.615545252s trace:
https://jaeger.dev.storj.io/trace/7db938728306fbdb
FAIL (2026-02-09 05:02:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.615580987s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....................X..X...................................] duration:
908.467718ms trace:
https://jaeger.dev.storj.io/trace/7fb5679f0d1238de
FAIL (2026-02-09 05:02:04 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 908.486142ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................X.X...................................] duration:
2.231834011s trace:
https://jaeger.dev.storj.io/trace/1e478b832adc7fb5
FAIL (2026-02-09 05:02:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
Feb 9 04:34:16.425527162 (588h38m54.309481351s) [upload=518198]: Starting upload
Feb 9 04:34:16.425614910 (588h38m54.30956911s) [upload=518198]: Done starting upload
Feb 9 04:34:16.425686379 (588h38m54.309640565s) [upload=518198]: Uploading segments...
Feb 9 04:34:16.425998866 (588h38m54.309953067s) [upload=518198]: Got next segment. Inline: false
Feb 9 04:34:16.426077295 (588h38m54.310031504s) [upload=518198]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 9 04:34:16.426084296 (588h38m54.310038505s) [upload=518198]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 9 04:34:16.539403201 (588h38m54.4233574s) [seg_pos={0 0}, upload=518198]: Begin upload segment...
Feb 9 04:34:16.539636372 (588h38m54.423590581s) [noise=true , address=145.79.227.44:4200 , piece_id=THUFQP3XAQBOE6BTDSRC2BOPHJBGLXSQNQKHAG3ZQWN6XL54745Q, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.540344248 (588h38m54.424298449s) [seg_pos={0 0}, upload=518198]: Done begin upload segment.
Feb 9 04:34:16.540380789 (588h38m54.424421242s) [noise=true , address=145.79.227.37:4588 , piece_id=GFCOSNHBNLRETLADAMBLCNXNORCBTAHA6QIPJ6FFKRGVSCUIAP3A, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.540813561 (588h38m54.424767771s) [noise=true , address=77.90.31.21:4897 , piece_id=NYYQ3XRMOFPQ76N7FVWX4ZMPHOLPIO55ZNYOYOMEY4AZ2FV7F3WA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.541105928 (588h38m54.425060128s) [noise=true , address=77.90.31.18:4826 , piece_id=MUTU2BT5YTCI7SWC3UHYYK4FQE5G7PNSYDRTDPKHJV66REC5XA3Q, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.541255031 (588h38m54.425209231s) [noise=true , address=145.79.227.19:4104 , piece_id=VA6C5GNCLDKT3UWJVGXYAR3HABMQFFJ73Y57ZEVGOPVZPN3U2YZQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.541771561 (588h38m54.425725763s) [noise=true , address=79.127.219.45:4868 , piece_id=F2DZHS5VN7IFGFZAFPKNIQ7LW37JH6NNQU76IKIUBVRPRWT36TSQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.542336894 (588h38m54.426291096s) [noise=true , address=77.90.31.10:4531 , piece_id=Y7RBZXSHFOOBTISMPH6VBAMV3ZZAZAAH75NYPZP64K4DOG6J7X7A, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.542856214 (588h38m54.426810421s) [noise=true , address=64.49.13.170:4725 , piece_id=FJ7LY7HY2RKPMN6VAP5D2MNXD33XZVJUXPSBK4FWBE3MV6D7CDPQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.544116137 (588h38m54.428070342s) [noise=true , address=79.127.219.41:4931 , piece_id=TJOGRTPRIXNCIZH37FMBANDIPNZGLED2QOOPURPV7NY3E3U7LJ2Q, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.544608261 (588h38m54.428562461s) [noise=true , address=145.79.227.36:4522 , piece_id=H4BPWZKCOJSM32VGBOLW5OCYDVSMT2YYOPNPKX37XIN64SSBW55A, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.545047288 (588h38m54.429001497s) [noise=true , address=145.79.227.34:4932 , piece_id=6P4HILBHZKG5PSO5UKRDZ2L2KQ3MNCGSMNTT4AKYROOM3G5XFALA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.545675258 (588h38m54.429629466s) [noise=true , address=77.90.31.19:4857 , piece_id=XM2V566CPCILX4U3WMMATKJXNI52DCPN6ZHEYWN6XJLHN3SAAB3A, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.546288192 (588h38m54.430242394s) [noise=true , address=77.90.31.13:4653 , piece_id=733247IC2BSFCHBPRLYSOAM3YJJM73HT2K3FAHQDJA7EUO4UO2FA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.546820663 (588h38m54.430774863s) [noise=true , address=77.90.31.15:4710 , piece_id=YE25JHYBLJIMLAQGC57XQBS3NT44QVQL6Y3LLC6IMJA3UBZMCD2Q, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.547480432 (588h38m54.431434642s) [noise=true , address=145.79.227.41:4098 , piece_id=HHLOOAW2HISRBZBNXHB3IURICY4TKH4TI7LUZ3YYHP47U5FI5EGQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.548092704 (588h38m54.432046906s) [noise=true , address=79.127.219.46:4903 , piece_id=5AB6ESAUL35HINFZQ7FSVBCJYLY5QOPNF6YIQUH6YLJXZKHQ5IAQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.548485216 (588h38m54.432439419s) [noise=true , address=79.127.219.38:4825 , piece_id=4KWAIHWK76P2GIJDP7SNDJOLHVGJKHXI5M2J34D2JKDB2J57A3OA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.548835080 (588h38m54.432789285s) [noise=true , address=77.90.31.17:4804 , piece_id=FZTRYSIGA7SGLHAIV44N4A3LTVBRH3FIXY2PTT2KFOSBAYONTSTA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:17.090871348 (588h38m54.974825556s) [upload=518198]: Next returned nil segment
Feb 9 04:34:17.090889807 (588h38m54.974844013s) [upload=518198]: Waiting for error group managing segments...
Feb 9 04:34:17.239393228 (588h38m55.123347429s) [noise=true , address=79.127.219.45:4868 , piece_id=F2DZHS5VN7IFGFZAFPKNIQ7LW37JH6NNQU76IKIUBVRPRWT36TSQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.253828118 (588h38m55.137782306s) [noise=true , address=79.127.219.41:4931 , piece_id=TJOGRTPRIXNCIZH37FMBANDIPNZGLED2QOOPURPV7NY3E3U7LJ2Q, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.286316140 (588h38m55.170270342s) [noise=true , address=77.90.31.13:4653 , piece_id=733247IC2BSFCHBPRLYSOAM3YJJM73HT2K3FAHQDJA7EUO4UO2FA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.287554811 (588h38m55.171509033s) [noise=true , address=77.90.31.18:4826 , piece_id=MUTU2BT5YTCI7SWC3UHYYK4FQE5G7PNSYDRTDPKHJV66REC5XA3Q, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.301101288 (588h38m55.185055487s) [noise=true , address=145.79.227.19:4104 , piece_id=VA6C5GNCLDKT3UWJVGXYAR3HABMQFFJ73Y57ZEVGOPVZPN3U2YZQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.305761625 (588h38m55.189715825s) [noise=true , address=79.127.219.46:4903 , piece_id=5AB6ESAUL35HINFZQ7FSVBCJYLY5QOPNF6YIQUH6YLJXZKHQ5IAQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.310622363 (588h38m55.194576563s) [noise=true , address=79.127.219.38:4825 , piece_id=4KWAIHWK76P2GIJDP7SNDJOLHVGJKHXI5M2J34D2JKDB2J57A3OA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.316237374 (588h38m55.200191573s) [noise=true , address=145.79.227.44:4200 , piece_id=THUFQP3XAQBOE6BTDSRC2BOPHJBGLXSQNQKHAG3ZQWN6XL54745Q, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.318039841 (588h38m55.20199404s) [noise=true , address=77.90.31.15:4710 , piece_id=YE25JHYBLJIMLAQGC57XQBS3NT44QVQL6Y3LLC6IMJA3UBZMCD2Q, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.354304865 (588h38m55.238259118s) [noise=true , address=77.90.31.17:4804 , piece_id=FZTRYSIGA7SGLHAIV44N4A3LTVBRH3FIXY2PTT2KFOSBAYONTSTA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.456416308 (588h38m55.340370507s) [noise=true , address=145.79.227.34:4932 , piece_id=6P4HILBHZKG5PSO5UKRDZ2L2KQ3MNCGSMNTT4AKYROOM3G5XFALA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:18.064577119 (588h38m55.948531321s) [noise=true , address=64.49.13.170:4725 , piece_id=FJ7LY7HY2RKPMN6VAP5D2MNXD33XZVJUXPSBK4FWBE3MV6D7CDPQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:18.377401016 (588h38m56.261355219s) [noise=true , address=77.90.31.19:4857 , piece_id=XM2V566CPCILX4U3WMMATKJXNI52DCPN6ZHEYWN6XJLHN3SAAB3A, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:35:16.136794989 (588h39m54.020749194s) [noise=true , address=77.90.31.10:4531 , piece_id=Y7RBZXSHFOOBTISMPH6VBAMV3ZZAZAAH75NYPZP64K4DOG6J7X7A, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:1AvSrhj1nHWLBJrw1SKacnjPTi9g4AzjqKvVFoxtzAdJFYAfQt, address:77.90.31.10:4531): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.136834341 (588h39m54.020788544s) [noise=true , address=145.79.227.41:4098 , piece_id=HHLOOAW2HISRBZBNXHB3IURICY4TKH4TI7LUZ3YYHP47U5FI5EGQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:1SoFNdwfh4p5tXUkpfjQaev6tdA2ZdhsXUFvZNBsy5QrJ9zdqb, address:145.79.227.41:4098): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.137014242 (588h39m54.020968441s) [noise=true , address=145.79.227.37:4588 , piece_id=GFCOSNHBNLRETLADAMBLCNXNORCBTAHA6QIPJ6FFKRGVSCUIAP3A, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:128rq4MaAx7kfBVrh2x8WWRi8BgD7nRoJSEpJjMQvPmKgtDtVnK, address:145.79.227.37:4588): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.137049764 (588h39m54.021003964s) [noise=true , address=77.90.31.21:4897 , piece_id=NYYQ3XRMOFPQ76N7FVWX4ZMPHOLPIO55ZNYOYOMEY4AZ2FV7F3WA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:12tc8y4NueDCMnENN3Arf4ecmGAhobx4HGuCLpidZmiDFzbGu6x, address:77.90.31.21:4897): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.137189462 (588h39m54.02114367s) [noise=true , address=145.79.227.36:4522 , piece_id=H4BPWZKCOJSM32VGBOLW5OCYDVSMT2YYOPNPKX37XIN64SSBW55A, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:1zGZe6e37uop4y622fU1GbspMPVEZURS3FbMrf7je6VT9TYGFr, address:145.79.227.36:4522): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.137220651 (588h39m54.02117484s) [seg_pos={0 0}, upload=518198]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Feb 9 04:36:32.038895323 (588h41m9.922849524s) [upload=518198]: Done uploading segments...
Feb 9 04:36:32.038928075 (588h41m9.922882286s) [upload=518198]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.231933361s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................X..X..................................] duration:
2.411055358s trace:
https://jaeger.dev.storj.io/trace/2367fd09c0ecbab9
FAIL (2026-02-09 05:02:02 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
upload segment...
Feb 9 04:35:22.700329253 (588h40m0.584283452s) [seg_pos={0 0}, upload=518203]: Done begin upload segment.
Feb 9 04:35:22.700460624 (588h40m0.584414826s) [noise=true , address=77.90.31.14:4664 , piece_id=NZ2S43RG6L4ISS4D5TNSLKPCOMG7NRLI5FJH5STGMZ4TSRHSCOXA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.700988853 (588h40m0.584943055s) [noise=true , address=77.90.31.21:4877 , piece_id=KSXI5U3CRHKFFRLEB6M6T4NPGNK3GQM2WWT5JFKGJRHMDTJR4SHA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.701851965 (588h40m0.585806164s) [noise=true , address=79.127.219.41:4930 , piece_id=AWPOIHF6TLMVHNHYDJKVY5OSVWT22NG33EDOHRCK23323ZMQKFGQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702127082 (588h40m0.586081284s) [noise=true , address=77.90.31.21:4880 , piece_id=C6FMFW55AYOLQDP7KLSLOJTC5G5TRXVQED7ZVEQ6NIAJJYOL7Z2Q, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702346744 (588h40m0.586300954s) [noise=true , address=45.91.116.7:4024 , piece_id=CRA3XDRATYDKNGI55D5WFKDMKZ7VSG44NHGW7N22KMEHKUNAP4QA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702509793 (588h40m0.586463991s) [noise=true , address=77.90.31.19:4844 , piece_id=6RD5ZBHA3MGQAXUUHQ3KVTYFZXMQRJMKV2DXWFEMXNS3CJEGD42A, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702814626 (588h40m0.586768829s) [noise=true , address=45.91.116.12:4228 , piece_id=DEDCYXMNUKHEIQKD2FLLJFPOTFS7QINLNFYD4SIKL42YYOSRHI4Q, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702824399 (588h40m0.586778657s) [noise=true , address=77.90.31.14:4661 , piece_id=RMTP6QMQTSCXHDWHAHPRDNL26P5X2ZQ5NVYFNAVTELPA6THRM5AA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.703210558 (588h40m0.587164761s) [noise=true , address=77.90.31.11:4556 , piece_id=SIC2PM67TPSE6ETWMINBVBKJGJFP2I5XK6LLTAZXG6TZOFYZ33EQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.703279077 (588h40m0.587233278s) [noise=true , address=77.90.31.17:4773 , piece_id=XUD5VZGZSZAH72H4QS5FSW5BGKDWB62YDNDD3Z2VGN7EHLFJBYRQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.703549671 (588h40m0.58750387s) [noise=true , address=77.90.31.21:4878 , piece_id=NS47GXLZ2KCDRIDZOAIHBGO2U2PYS2HOBOSYW43WLISJWRCAPFNQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.703771063 (588h40m0.587725273s) [noise=true , address=79.127.219.41:4926 , piece_id=GKWSUWOKER27WJNW3MHWD6ZZATXPV4ZDC4VNBRTYOHXJMA33UBBQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.704208043 (588h40m0.588162245s) [noise=true , address=79.127.219.45:4856 , piece_id=IXWYK73XOZSITPY2PO6BBAAWVTC76CWJFFFWKXSNOGYFK6LCWQPQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.704629165 (588h40m0.588583365s) [noise=true , address=45.91.116.12:4244 , piece_id=N3AGJ5VS5OK6SGM6CXOELXTPI5MTEECVHFPEA2HE6HYMGEFJVI3Q, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.705058487 (588h40m0.58901269s) [noise=true , address=77.90.31.19:4842 , piece_id=DC3RQKVSIEEHNZQM4EQL54S6IHO6ORZQOITSE34LD2VJSWMR4QYA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.705566550 (588h40m0.589520752s) [noise=true , address=77.90.31.15:4697 , piece_id=AQ5RTDKKEL553KLEDMNB2S7S2YMMREYA7CFNUI7M2VHGN3ISG27Q, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:23.857509627 (588h40m1.741463833s) [upload=518203]: Next returned nil segment
Feb 9 04:35:23.857528736 (588h40m1.741482946s) [upload=518203]: Waiting for error group managing segments...
Feb 9 04:35:24.037977395 (588h40m1.921931594s) [noise=true , address=79.127.219.41:4930 , piece_id=AWPOIHF6TLMVHNHYDJKVY5OSVWT22NG33EDOHRCK23323ZMQKFGQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:26.272463124 (588h40m4.156417324s) [noise=true , address=79.127.219.41:4926 , piece_id=GKWSUWOKER27WJNW3MHWD6ZZATXPV4ZDC4VNBRTYOHXJMA33UBBQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:34.140028924 (588h40m12.023983124s) [noise=true , address=77.90.31.15:4697 , piece_id=AQ5RTDKKEL553KLEDMNB2S7S2YMMREYA7CFNUI7M2VHGN3ISG27Q, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:37.255393413 (588h40m15.139347624s) [noise=true , address=77.90.31.14:4661 , piece_id=RMTP6QMQTSCXHDWHAHPRDNL26P5X2ZQ5NVYFNAVTELPA6THRM5AA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:41.697263539 (588h40m19.581217739s) [noise=true , address=77.90.31.19:4842 , piece_id=DC3RQKVSIEEHNZQM4EQL54S6IHO6ORZQOITSE34LD2VJSWMR4QYA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:41.901811216 (588h40m19.785765419s) [noise=true , address=45.91.116.12:4244 , piece_id=N3AGJ5VS5OK6SGM6CXOELXTPI5MTEECVHFPEA2HE6HYMGEFJVI3Q, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:47.044981722 (588h40m24.928935927s) [noise=true , address=45.91.116.7:4024 , piece_id=CRA3XDRATYDKNGI55D5WFKDMKZ7VSG44NHGW7N22KMEHKUNAP4QA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: rpc: dial tcp 45.91.116.7:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045035145 (588h40m24.928989353s) [noise=true , address=79.127.219.45:4856 , piece_id=IXWYK73XOZSITPY2PO6BBAAWVTC76CWJFFFWKXSNOGYFK6LCWQPQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:121kT5cTDuopYFWfFYdmdj6QBRJMybyL7G2ow4mFeZFEH3QVxE6, address:79.127.219.45:4856): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:47.045265584 (588h40m24.92921979s) [noise=true , address=77.90.31.14:4664 , piece_id=NZ2S43RG6L4ISS4D5TNSLKPCOMG7NRLI5FJH5STGMZ4TSRHSCOXA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 139.59.144.22:42286->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045305927 (588h40m24.929260132s) [noise=true , address=77.90.31.21:4878 , piece_id=NS47GXLZ2KCDRIDZOAIHBGO2U2PYS2HOBOSYW43WLISJWRCAPFNQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: read tcp 139.59.144.22:37798->77.90.31.21:4878: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045384440 (588h40m24.929338638s) [noise=true , address=77.90.31.17:4773 , piece_id=XUD5VZGZSZAH72H4QS5FSW5BGKDWB62YDNDD3Z2VGN7EHLFJBYRQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 139.59.144.22:48366->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045477234 (588h40m24.929431437s) [noise=true , address=77.90.31.21:4880 , piece_id=C6FMFW55AYOLQDP7KLSLOJTC5G5TRXVQED7ZVEQ6NIAJJYOL7Z2Q, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: write tcp 139.59.144.22:44174->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045498918 (588h40m24.929453118s) [noise=true , address=77.90.31.21:4877 , piece_id=KSXI5U3CRHKFFRLEB6M6T4NPGNK3GQM2WWT5JFKGJRHMDTJR4SHA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 139.59.144.22:38330->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045701538 (588h40m24.929655736s) [noise=true , address=77.90.31.19:4844 , piece_id=6RD5ZBHA3MGQAXUUHQ3KVTYFZXMQRJMKV2DXWFEMXNS3CJEGD42A, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: read tcp 139.59.144.22:42234->77.90.31.19:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045832097 (588h40m24.929786297s) [noise=true , address=77.90.31.11:4556 , piece_id=SIC2PM67TPSE6ETWMINBVBKJGJFP2I5XK6LLTAZXG6TZOFYZ33EQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:39978->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045842467 (588h40m24.929796668s) [noise=true , address=45.91.116.12:4228 , piece_id=DEDCYXMNUKHEIQKD2FLLJFPOTFS7QINLNFYD4SIKL42YYOSRHI4Q, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, address:45.91.116.12:4228): protocol: read tcp 139.59.144.22:55958->45.91.116.12:4228: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045865196 (588h40m24.929819381s) [seg_pos={0 0}, upload=518203]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Feb 9 04:35:59.996828134 (588h40m37.88078234s) [upload=518203]: Done uploading segments...
Feb 9 04:35:59.996890408 (588h40m37.880844607s) [upload=518203]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 2.411145677s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
4 out of 60 is failed (threshold=3) [.....................XX..XX.................................] duration:
1.59468516s trace:
https://jaeger.dev.storj.io/trace/75241c521a7f3dcb
FAIL (2026-02-09 05:01:57 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 1.594743432s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.................XX..X......................................] duration:
1.251481402s trace:
https://jaeger.dev.storj.io/trace/3aaeb3002c06c37e
FAIL (2026-02-09 05:01:55 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 51)
Duration: 1.251511708s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.......................X..X.................................] duration:
2.843439789s trace:
https://jaeger.dev.storj.io/trace/46fbafc4cec1829c
FAIL (2026-02-09 05:01:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.843467967s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [...............X..XX.................................X......] duration:
2.804562995s trace:
https://jaeger.dev.storj.io/trace/7db938728306fbdb
FAIL (2026-02-09 05:01:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.80459012s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [....................XXX.XX..................................] duration:
1.686481809s trace:
https://jaeger.dev.storj.io/trace/52e03ea49baff327
FAIL (2026-02-09 05:01:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.686514484s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
343.219899ms target:
https://version.storj.io/ trace:
https://jaeger.dev.storj.io/trace/65dcc1a2b145167b
FAIL (2026-02-09 05:01:41 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 343.284ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [...................XX..X....................................] duration:
963.157232ms trace:
https://jaeger.dev.storj.io/trace/10c99c609f21fd50
FAIL (2026-02-09 05:01:40 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 963.185465ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
288.710588ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/2b03593df22bf5a6
FAIL (2026-02-09 05:01:40 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout
Duration: 288.735018ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
2 out of 60 is failed (threshold=3) [....................X..X....................................] duration:
908.807085ms trace:
https://jaeger.dev.storj.io/trace/2851863ba19d723d
FAIL (2026-02-09 05:01:39 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 908.835702ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
3 out of 60 is failed (threshold=3) [......................X.XX..................................] duration:
3.020260016s trace:
https://jaeger.dev.storj.io/trace/2545ffb6627030f4
FAIL (2026-02-09 05:01:38 UTC)
orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 3.020289987s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..................XX..X.....................................] duration:
1.422649395s trace:
https://jaeger.dev.storj.io/trace/3aaeb3002c06c37e
FAIL (2026-02-09 05:01:26 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 51)
Duration: 1.422683615s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
336.107803ms target:
https://version.storj.io/ trace:
https://jaeger.dev.storj.io/trace/65dcc1a2b145167b
FAIL (2026-02-09 05:01:26 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 336.124285ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
4 out of 60 is failed (threshold=3) [...................XX..XX...................................] duration:
2.185490786s trace:
https://jaeger.dev.storj.io/trace/3417326bc88628d0
FAIL (2026-02-09 05:01:21 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.185521671s
core
uplink
us1
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [....................X.X.....................................] duration:
532.912584ms remote:
"tcp:34.2.157.232:443" target:
https://ap1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/96628cddd17f5b6
FAIL (2026-02-09 05:01:16 UTC)
orbiter warning: Get "https://ap1.storj.io/login": context deadline exceeded
Duration: 532.938915ms
core
satellite
ap1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
297.068329ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/2b03593df22bf5a6
FAIL (2026-02-09 05:01:13 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout
Duration: 297.096913ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
343.164416ms target:
https://version.storj.io/ trace:
https://jaeger.dev.storj.io/trace/65dcc1a2b145167b
FAIL (2026-02-09 05:01:08 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 343.210016ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
2 out of 60 is failed (threshold=3) [......................X..X..................................] duration:
870.935313ms trace:
https://jaeger.dev.storj.io/trace/7fb5679f0d1238de
FAIL (2026-02-09 05:01:05 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 870.961866ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [................X..XX.................................X.....] duration:
2.847033865s trace:
https://jaeger.dev.storj.io/trace/7db938728306fbdb
FAIL (2026-02-09 05:01:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.847071013s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [........................X.X.................................] duration:
446.17495ms trace:
https://jaeger.dev.storj.io/trace/7e5469f7055ee287
FAIL (2026-02-09 05:01:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
e...
Feb 9 04:34:35.705893606 (588h39m13.589847811s) [noise=true , address=185.167.204.206:4066 , piece_id=I6GSQDXZSHWF7POYBOQYLJU2QKDITQDFVE3P2KOBHEKSVSDKJIWA, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.707890134 (588h39m13.591844338s) [noise=true , address=89.149.201.147:4015 , piece_id=4DAM72SN2235KI22UTCVVVBRVUANAWP3D5YVQJTHNV3KB563ZI6A, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.709078016 (588h39m13.593032221s) [noise=true , address=185.167.204.206:4060 , piece_id=GME5JPQ4VVG2JII35UMC6XHEAKUWYASOJ63QFLO5NIJ7LIRL3ZWA, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.710195507 (588h39m13.594149715s) [noise=true , address=185.167.204.206:4065 , piece_id=3KFJZZ4SYCKW45PEGYY6EANCLFAZDO4DGVQCD6OBWDVTOAHKUDRQ, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.711495198 (588h39m13.5954494s) [noise=true , address=81.171.14.70:4032 , piece_id=G4SCAKGMP7M5VCXNK2BIN7SVOYOF5JCJQSUWCOIC2WR5SYYVX5SA, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.712109524 (588h39m13.596063726s) [noise=true , address=185.167.204.205:4019 , piece_id=3HIP6GSZ3Y7EEW3YBNEXSMURSNONFMAP6J3V5QLQ4JTEXY7K2F7Q, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.712699464 (588h39m13.596653666s) [noise=true , address=89.149.201.147:4016 , piece_id=IUY7UBPUW3IC5KBLOGWRTR4QOED5FN6BLCJINEHSW7SO5E7W27PA, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.713261874 (588h39m13.597216076s) [noise=true , address=89.149.201.147:4007 , piece_id=DQLCJPMI5U6MFTBTTUZAADXHFOGI4RNDYOSV6VOPPIJ2ZNQIXHQQ, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:36.430571544 (588h39m14.314525751s) [upload=518201]: Next returned nil segment
Feb 9 04:34:36.430612826 (588h39m14.314567032s) [upload=518201]: Waiting for error group managing segments...
Feb 9 04:34:37.285542315 (588h39m15.169496516s) [noise=true , address=89.149.201.147:4020 , piece_id=AT4BOPLHXF6T72VGDVV3MN7CSW6HXDI4MHPKDMHHQYDVTRY2XD7Q, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:37.352144984 (588h39m15.23609919s) [noise=true , address=89.149.201.147:4011 , piece_id=JUOZTOX2S7JFJDPU632FXSQUGPW3NIKYOYKB7DFAZHHARFQDJKWQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:37.474326245 (588h39m15.358280455s) [noise=true , address=81.171.14.70:4046 , piece_id=CYD2R5P2GCFCKXOXG5TPQFU7SB66JN24CWQKTXAL2EMMZTMEJNNA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:37.706787293 (588h39m15.590741558s) [noise=true , address=81.171.14.70:4035 , piece_id=LYHOYHAN6CJOXUK7PUL7AJGQDOQMD234FY7LO3TKF7HCQK345NCQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:37.974945056 (588h39m15.858899257s) [noise=true , address=185.167.204.206:4057 , piece_id=NQ2DR2IDYK5BR77YUC7G2A2YBVMXJ6CXZ2GQIP5332RSPHUOT3SQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:38.080441216 (588h39m15.964395418s) [noise=true , address=81.171.14.70:4045 , piece_id=G35PNKGJFVLFMJ4OUHIEJTXLHD35AYSSXCNNLIVMJPAV2ZVL4HGQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:38.486602188 (588h39m16.370556388s) [noise=true , address=185.167.204.206:4061 , piece_id=OT2TOUL5ARMB476CHGPNP6NXZIUXJRHS4MTKS24LH2XYYE5CTUHQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:38.740717181 (588h39m16.62467138s) [noise=true , address=185.167.204.206:4047 , piece_id=UH4Y5DREYE4ROEIW77ZSQHFBCHWBEFMBJ4KI27MQV4LWZI47MPLQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:38.773718899 (588h39m16.657673102s) [noise=true , address=89.149.201.147:4007 , piece_id=DQLCJPMI5U6MFTBTTUZAADXHFOGI4RNDYOSV6VOPPIJ2ZNQIXHQQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:38.966510172 (588h39m16.850464373s) [noise=true , address=185.167.204.205:4012 , piece_id=EOCCFH625F326UVQXAFOH354JRJBLB74ZZDCBONQAIINRJG6UW4A, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:39.816146031 (588h39m17.700100235s) [noise=true , address=81.171.14.70:4040 , piece_id=PTNV3ZZLLZLQNUHVTOD654PRBEMFLV6T5G3ULMZW24NBXKMFKZVA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:41.016111070 (588h39m18.900065269s) [noise=true , address=81.171.14.70:4034 , piece_id=FDKTEAXHBMFPXNLZCAB4NDXXKVUMHMFRX6QXBLJQ3MROXVINDPPQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:42.460071941 (588h39m20.344026131s) [noise=true , address=81.171.14.70:4044 , piece_id=N4J4UTUGGXUNQCMXIMNS6HV7XXGO3OPS6HFCMPY5SVK2HFNBXR4Q, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:48.752914118 (588h39m26.636868321s) [noise=true , address=185.167.204.206:4060 , piece_id=GME5JPQ4VVG2JII35UMC6XHEAKUWYASOJ63QFLO5NIJ7LIRL3ZWA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:58.087687791 (588h39m35.971641979s) [noise=true , address=185.167.204.206:4065 , piece_id=3KFJZZ4SYCKW45PEGYY6EANCLFAZDO4DGVQCD6OBWDVTOAHKUDRQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:59.132504306 (588h39m37.016458494s) [noise=true , address=81.171.14.70:4041 , piece_id=CODI7DQA7CC66YPS56IUNXMFCFHEXX4SYCEEYGF4TYXXQFEM7WHA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:35:01.532538593 (588h39m39.4164928s) [noise=true , address=89.149.201.147:4016 , piece_id=IUY7UBPUW3IC5KBLOGWRTR4QOED5FN6BLCJINEHSW7SO5E7W27PA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:35:03.766981205 (588h39m41.650935408s) [noise=true , address=89.149.201.147:4015 , piece_id=4DAM72SN2235KI22UTCVVVBRVUANAWP3D5YVQJTHNV3KB563ZI6A, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:35:09.427240351 (588h39m47.311194536s) [noise=true , address=89.149.201.147:4013 , piece_id=6OUUA7ZOINHMDPGRJ44HZDTEJS522BTSBM3F42IESIL734DEHUUA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:35:14.481090345 (588h39m52.365044556s) [noise=true , address=185.167.204.205:4026 , piece_id=VGMKJEZ65M7SUDO7BHITUSAKOQCIQJGI3MNV6XLTIFG7X3JPDK2Q, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:35:16.756347933 (588h39m54.640302133s) [noise=true , address=81.171.14.70:4028 , piece_id=AMO232UCVQTFA6O62VUWDXVGODDHPQRFMCGRAMNGARZHCGEARQDA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:35:34.283098105 (588h40m12.167052314s) [noise=true , address=185.167.204.205:4003 , piece_id=UZN6VHXQ4CMZ3TOKA4OPTGONEYNMAKPW6Z3W32D7XJTN625JDZ7A, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:1CjWvvJ4FdZc41sRgsQFWz4Y1ATZKgzsvDcUTmBSrCC15ekCmF, address:185.167.204.205:4003): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:34.283115609 (588h40m12.167069803s) [noise=true , address=81.171.14.70:4032 , piece_id=G4SCAKGMP7M5VCXNK2BIN7SVOYOF5JCJQSUWCOIC2WR5SYYVX5SA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 139.59.144.22:52900->81.171.14.70:4032: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:34.283347033 (588h40m12.167301242s) [noise=true , address=185.167.204.206:4069 , piece_id=LGOOCFX4YC53O5DH6IUFYXKFNJNZTM7BFLA7KLU3HYKARMLZ2WOA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:34.283388338 (588h40m12.167342536s) [noise=true , address=89.149.201.147:4009 , piece_id=4P2UU2WFURMXZQFYKEUTDD3NGXSFGGPQ3ACEJCYGCYXHHGDERKNA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:1sbZWJFMVsppCpbFVfn33bx5uGkhtFpzJm8a2i8DoAfQNsWY5F, address:89.149.201.147:4009): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:34.283661254 (588h40m12.167615456s) [noise=true , address=89.149.201.147:4014 , piece_id=D6EFC33PCPWUACSSEO4INMWDG557CZYQYFAAOI3JFFPBPLRFXNTQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 139.59.144.22:53384->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:34.284010218 (588h40m12.167964417s) [noise=true , address=185.167.204.205:4019 , piece_id=3HIP6GSZ3Y7EEW3YBNEXSMURSNONFMAP6J3V5QLQ4JTEXY7K2F7Q, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:12CwWyCCjbNNuuSxpk9PhiWxG2yvuT19gDyJhezMfUEez6LrpDj, address:185.167.204.205:4019): protocol: read tcp 139.59.144.22:58778->185.167.204.205:4019: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:34.284176217 (588h40m12.168130427s) [noise=true , address=185.167.204.206:4066 , piece_id=I6GSQDXZSHWF7POYBOQYLJU2QKDITQDFVE3P2KOBHEKSVSDKJIWA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: read tcp 139.59.144.22:38920->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:34.284215153 (588h40m12.168169349s) [seg_pos={0 0}, upload=518201]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Feb 9 04:35:34.334736863 (588h40m12.218691063s) [upload=518201]: Done uploading segments...
Feb 9 04:35:34.334794137 (588h40m12.218748348s) [upload=518201]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 446.208072ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...................XX..X....................................] duration:
1.024574604s trace:
https://jaeger.dev.storj.io/trace/3aaeb3002c06c37e
FAIL (2026-02-09 05:00:57 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 51)
Duration: 1.024602477s
core
uplink
eu1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
4 out of 60 is failed (threshold=3) [......................XX..XX................................] duration:
1.705003708s trace:
https://jaeger.dev.storj.io/trace/75241c521a7f3dcb
FAIL (2026-02-09 05:00:56 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 1.705047748s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................X..X.................................] duration:
2.462074164s trace:
https://jaeger.dev.storj.io/trace/2367fd09c0ecbab9
FAIL (2026-02-09 05:00:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
upload segment...
Feb 9 04:35:22.700329253 (588h40m0.584283452s) [seg_pos={0 0}, upload=518203]: Done begin upload segment.
Feb 9 04:35:22.700460624 (588h40m0.584414826s) [noise=true , address=77.90.31.14:4664 , piece_id=NZ2S43RG6L4ISS4D5TNSLKPCOMG7NRLI5FJH5STGMZ4TSRHSCOXA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.700988853 (588h40m0.584943055s) [noise=true , address=77.90.31.21:4877 , piece_id=KSXI5U3CRHKFFRLEB6M6T4NPGNK3GQM2WWT5JFKGJRHMDTJR4SHA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.701851965 (588h40m0.585806164s) [noise=true , address=79.127.219.41:4930 , piece_id=AWPOIHF6TLMVHNHYDJKVY5OSVWT22NG33EDOHRCK23323ZMQKFGQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702127082 (588h40m0.586081284s) [noise=true , address=77.90.31.21:4880 , piece_id=C6FMFW55AYOLQDP7KLSLOJTC5G5TRXVQED7ZVEQ6NIAJJYOL7Z2Q, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702346744 (588h40m0.586300954s) [noise=true , address=45.91.116.7:4024 , piece_id=CRA3XDRATYDKNGI55D5WFKDMKZ7VSG44NHGW7N22KMEHKUNAP4QA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702509793 (588h40m0.586463991s) [noise=true , address=77.90.31.19:4844 , piece_id=6RD5ZBHA3MGQAXUUHQ3KVTYFZXMQRJMKV2DXWFEMXNS3CJEGD42A, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702814626 (588h40m0.586768829s) [noise=true , address=45.91.116.12:4228 , piece_id=DEDCYXMNUKHEIQKD2FLLJFPOTFS7QINLNFYD4SIKL42YYOSRHI4Q, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702824399 (588h40m0.586778657s) [noise=true , address=77.90.31.14:4661 , piece_id=RMTP6QMQTSCXHDWHAHPRDNL26P5X2ZQ5NVYFNAVTELPA6THRM5AA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.703210558 (588h40m0.587164761s) [noise=true , address=77.90.31.11:4556 , piece_id=SIC2PM67TPSE6ETWMINBVBKJGJFP2I5XK6LLTAZXG6TZOFYZ33EQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.703279077 (588h40m0.587233278s) [noise=true , address=77.90.31.17:4773 , piece_id=XUD5VZGZSZAH72H4QS5FSW5BGKDWB62YDNDD3Z2VGN7EHLFJBYRQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.703549671 (588h40m0.58750387s) [noise=true , address=77.90.31.21:4878 , piece_id=NS47GXLZ2KCDRIDZOAIHBGO2U2PYS2HOBOSYW43WLISJWRCAPFNQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.703771063 (588h40m0.587725273s) [noise=true , address=79.127.219.41:4926 , piece_id=GKWSUWOKER27WJNW3MHWD6ZZATXPV4ZDC4VNBRTYOHXJMA33UBBQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.704208043 (588h40m0.588162245s) [noise=true , address=79.127.219.45:4856 , piece_id=IXWYK73XOZSITPY2PO6BBAAWVTC76CWJFFFWKXSNOGYFK6LCWQPQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.704629165 (588h40m0.588583365s) [noise=true , address=45.91.116.12:4244 , piece_id=N3AGJ5VS5OK6SGM6CXOELXTPI5MTEECVHFPEA2HE6HYMGEFJVI3Q, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.705058487 (588h40m0.58901269s) [noise=true , address=77.90.31.19:4842 , piece_id=DC3RQKVSIEEHNZQM4EQL54S6IHO6ORZQOITSE34LD2VJSWMR4QYA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.705566550 (588h40m0.589520752s) [noise=true , address=77.90.31.15:4697 , piece_id=AQ5RTDKKEL553KLEDMNB2S7S2YMMREYA7CFNUI7M2VHGN3ISG27Q, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:23.857509627 (588h40m1.741463833s) [upload=518203]: Next returned nil segment
Feb 9 04:35:23.857528736 (588h40m1.741482946s) [upload=518203]: Waiting for error group managing segments...
Feb 9 04:35:24.037977395 (588h40m1.921931594s) [noise=true , address=79.127.219.41:4930 , piece_id=AWPOIHF6TLMVHNHYDJKVY5OSVWT22NG33EDOHRCK23323ZMQKFGQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:26.272463124 (588h40m4.156417324s) [noise=true , address=79.127.219.41:4926 , piece_id=GKWSUWOKER27WJNW3MHWD6ZZATXPV4ZDC4VNBRTYOHXJMA33UBBQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:34.140028924 (588h40m12.023983124s) [noise=true , address=77.90.31.15:4697 , piece_id=AQ5RTDKKEL553KLEDMNB2S7S2YMMREYA7CFNUI7M2VHGN3ISG27Q, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:37.255393413 (588h40m15.139347624s) [noise=true , address=77.90.31.14:4661 , piece_id=RMTP6QMQTSCXHDWHAHPRDNL26P5X2ZQ5NVYFNAVTELPA6THRM5AA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:41.697263539 (588h40m19.581217739s) [noise=true , address=77.90.31.19:4842 , piece_id=DC3RQKVSIEEHNZQM4EQL54S6IHO6ORZQOITSE34LD2VJSWMR4QYA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:41.901811216 (588h40m19.785765419s) [noise=true , address=45.91.116.12:4244 , piece_id=N3AGJ5VS5OK6SGM6CXOELXTPI5MTEECVHFPEA2HE6HYMGEFJVI3Q, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:47.044981722 (588h40m24.928935927s) [noise=true , address=45.91.116.7:4024 , piece_id=CRA3XDRATYDKNGI55D5WFKDMKZ7VSG44NHGW7N22KMEHKUNAP4QA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: rpc: dial tcp 45.91.116.7:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045035145 (588h40m24.928989353s) [noise=true , address=79.127.219.45:4856 , piece_id=IXWYK73XOZSITPY2PO6BBAAWVTC76CWJFFFWKXSNOGYFK6LCWQPQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:121kT5cTDuopYFWfFYdmdj6QBRJMybyL7G2ow4mFeZFEH3QVxE6, address:79.127.219.45:4856): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:47.045265584 (588h40m24.92921979s) [noise=true , address=77.90.31.14:4664 , piece_id=NZ2S43RG6L4ISS4D5TNSLKPCOMG7NRLI5FJH5STGMZ4TSRHSCOXA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 139.59.144.22:42286->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045305927 (588h40m24.929260132s) [noise=true , address=77.90.31.21:4878 , piece_id=NS47GXLZ2KCDRIDZOAIHBGO2U2PYS2HOBOSYW43WLISJWRCAPFNQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: read tcp 139.59.144.22:37798->77.90.31.21:4878: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045384440 (588h40m24.929338638s) [noise=true , address=77.90.31.17:4773 , piece_id=XUD5VZGZSZAH72H4QS5FSW5BGKDWB62YDNDD3Z2VGN7EHLFJBYRQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 139.59.144.22:48366->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045477234 (588h40m24.929431437s) [noise=true , address=77.90.31.21:4880 , piece_id=C6FMFW55AYOLQDP7KLSLOJTC5G5TRXVQED7ZVEQ6NIAJJYOL7Z2Q, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: write tcp 139.59.144.22:44174->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045498918 (588h40m24.929453118s) [noise=true , address=77.90.31.21:4877 , piece_id=KSXI5U3CRHKFFRLEB6M6T4NPGNK3GQM2WWT5JFKGJRHMDTJR4SHA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 139.59.144.22:38330->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045701538 (588h40m24.929655736s) [noise=true , address=77.90.31.19:4844 , piece_id=6RD5ZBHA3MGQAXUUHQ3KVTYFZXMQRJMKV2DXWFEMXNS3CJEGD42A, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: read tcp 139.59.144.22:42234->77.90.31.19:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045832097 (588h40m24.929786297s) [noise=true , address=77.90.31.11:4556 , piece_id=SIC2PM67TPSE6ETWMINBVBKJGJFP2I5XK6LLTAZXG6TZOFYZ33EQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:39978->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045842467 (588h40m24.929796668s) [noise=true , address=45.91.116.12:4228 , piece_id=DEDCYXMNUKHEIQKD2FLLJFPOTFS7QINLNFYD4SIKL42YYOSRHI4Q, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, address:45.91.116.12:4228): protocol: read tcp 139.59.144.22:55958->45.91.116.12:4228: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045865196 (588h40m24.929819381s) [seg_pos={0 0}, upload=518203]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Feb 9 04:35:59.996828134 (588h40m37.88078234s) [upload=518203]: Done uploading segments...
Feb 9 04:35:59.996890408 (588h40m37.880844607s) [upload=518203]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 2.462099822s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [........................X..X................................] duration:
2.879887985s trace:
https://jaeger.dev.storj.io/trace/46fbafc4cec1829c
FAIL (2026-02-09 05:00:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.879910735s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................X.X..................................] duration:
2.595833281s trace:
https://jaeger.dev.storj.io/trace/1e478b832adc7fb5
FAIL (2026-02-09 05:00:52 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
Feb 9 04:34:16.425527162 (588h38m54.309481351s) [upload=518198]: Starting upload
Feb 9 04:34:16.425614910 (588h38m54.30956911s) [upload=518198]: Done starting upload
Feb 9 04:34:16.425686379 (588h38m54.309640565s) [upload=518198]: Uploading segments...
Feb 9 04:34:16.425998866 (588h38m54.309953067s) [upload=518198]: Got next segment. Inline: false
Feb 9 04:34:16.426077295 (588h38m54.310031504s) [upload=518198]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 9 04:34:16.426084296 (588h38m54.310038505s) [upload=518198]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 9 04:34:16.539403201 (588h38m54.4233574s) [seg_pos={0 0}, upload=518198]: Begin upload segment...
Feb 9 04:34:16.539636372 (588h38m54.423590581s) [noise=true , address=145.79.227.44:4200 , piece_id=THUFQP3XAQBOE6BTDSRC2BOPHJBGLXSQNQKHAG3ZQWN6XL54745Q, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.540344248 (588h38m54.424298449s) [seg_pos={0 0}, upload=518198]: Done begin upload segment.
Feb 9 04:34:16.540380789 (588h38m54.424421242s) [noise=true , address=145.79.227.37:4588 , piece_id=GFCOSNHBNLRETLADAMBLCNXNORCBTAHA6QIPJ6FFKRGVSCUIAP3A, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.540813561 (588h38m54.424767771s) [noise=true , address=77.90.31.21:4897 , piece_id=NYYQ3XRMOFPQ76N7FVWX4ZMPHOLPIO55ZNYOYOMEY4AZ2FV7F3WA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.541105928 (588h38m54.425060128s) [noise=true , address=77.90.31.18:4826 , piece_id=MUTU2BT5YTCI7SWC3UHYYK4FQE5G7PNSYDRTDPKHJV66REC5XA3Q, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.541255031 (588h38m54.425209231s) [noise=true , address=145.79.227.19:4104 , piece_id=VA6C5GNCLDKT3UWJVGXYAR3HABMQFFJ73Y57ZEVGOPVZPN3U2YZQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.541771561 (588h38m54.425725763s) [noise=true , address=79.127.219.45:4868 , piece_id=F2DZHS5VN7IFGFZAFPKNIQ7LW37JH6NNQU76IKIUBVRPRWT36TSQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.542336894 (588h38m54.426291096s) [noise=true , address=77.90.31.10:4531 , piece_id=Y7RBZXSHFOOBTISMPH6VBAMV3ZZAZAAH75NYPZP64K4DOG6J7X7A, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.542856214 (588h38m54.426810421s) [noise=true , address=64.49.13.170:4725 , piece_id=FJ7LY7HY2RKPMN6VAP5D2MNXD33XZVJUXPSBK4FWBE3MV6D7CDPQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.544116137 (588h38m54.428070342s) [noise=true , address=79.127.219.41:4931 , piece_id=TJOGRTPRIXNCIZH37FMBANDIPNZGLED2QOOPURPV7NY3E3U7LJ2Q, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.544608261 (588h38m54.428562461s) [noise=true , address=145.79.227.36:4522 , piece_id=H4BPWZKCOJSM32VGBOLW5OCYDVSMT2YYOPNPKX37XIN64SSBW55A, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.545047288 (588h38m54.429001497s) [noise=true , address=145.79.227.34:4932 , piece_id=6P4HILBHZKG5PSO5UKRDZ2L2KQ3MNCGSMNTT4AKYROOM3G5XFALA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.545675258 (588h38m54.429629466s) [noise=true , address=77.90.31.19:4857 , piece_id=XM2V566CPCILX4U3WMMATKJXNI52DCPN6ZHEYWN6XJLHN3SAAB3A, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.546288192 (588h38m54.430242394s) [noise=true , address=77.90.31.13:4653 , piece_id=733247IC2BSFCHBPRLYSOAM3YJJM73HT2K3FAHQDJA7EUO4UO2FA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.546820663 (588h38m54.430774863s) [noise=true , address=77.90.31.15:4710 , piece_id=YE25JHYBLJIMLAQGC57XQBS3NT44QVQL6Y3LLC6IMJA3UBZMCD2Q, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.547480432 (588h38m54.431434642s) [noise=true , address=145.79.227.41:4098 , piece_id=HHLOOAW2HISRBZBNXHB3IURICY4TKH4TI7LUZ3YYHP47U5FI5EGQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.548092704 (588h38m54.432046906s) [noise=true , address=79.127.219.46:4903 , piece_id=5AB6ESAUL35HINFZQ7FSVBCJYLY5QOPNF6YIQUH6YLJXZKHQ5IAQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.548485216 (588h38m54.432439419s) [noise=true , address=79.127.219.38:4825 , piece_id=4KWAIHWK76P2GIJDP7SNDJOLHVGJKHXI5M2J34D2JKDB2J57A3OA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.548835080 (588h38m54.432789285s) [noise=true , address=77.90.31.17:4804 , piece_id=FZTRYSIGA7SGLHAIV44N4A3LTVBRH3FIXY2PTT2KFOSBAYONTSTA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:17.090871348 (588h38m54.974825556s) [upload=518198]: Next returned nil segment
Feb 9 04:34:17.090889807 (588h38m54.974844013s) [upload=518198]: Waiting for error group managing segments...
Feb 9 04:34:17.239393228 (588h38m55.123347429s) [noise=true , address=79.127.219.45:4868 , piece_id=F2DZHS5VN7IFGFZAFPKNIQ7LW37JH6NNQU76IKIUBVRPRWT36TSQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.253828118 (588h38m55.137782306s) [noise=true , address=79.127.219.41:4931 , piece_id=TJOGRTPRIXNCIZH37FMBANDIPNZGLED2QOOPURPV7NY3E3U7LJ2Q, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.286316140 (588h38m55.170270342s) [noise=true , address=77.90.31.13:4653 , piece_id=733247IC2BSFCHBPRLYSOAM3YJJM73HT2K3FAHQDJA7EUO4UO2FA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.287554811 (588h38m55.171509033s) [noise=true , address=77.90.31.18:4826 , piece_id=MUTU2BT5YTCI7SWC3UHYYK4FQE5G7PNSYDRTDPKHJV66REC5XA3Q, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.301101288 (588h38m55.185055487s) [noise=true , address=145.79.227.19:4104 , piece_id=VA6C5GNCLDKT3UWJVGXYAR3HABMQFFJ73Y57ZEVGOPVZPN3U2YZQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.305761625 (588h38m55.189715825s) [noise=true , address=79.127.219.46:4903 , piece_id=5AB6ESAUL35HINFZQ7FSVBCJYLY5QOPNF6YIQUH6YLJXZKHQ5IAQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.310622363 (588h38m55.194576563s) [noise=true , address=79.127.219.38:4825 , piece_id=4KWAIHWK76P2GIJDP7SNDJOLHVGJKHXI5M2J34D2JKDB2J57A3OA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.316237374 (588h38m55.200191573s) [noise=true , address=145.79.227.44:4200 , piece_id=THUFQP3XAQBOE6BTDSRC2BOPHJBGLXSQNQKHAG3ZQWN6XL54745Q, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.318039841 (588h38m55.20199404s) [noise=true , address=77.90.31.15:4710 , piece_id=YE25JHYBLJIMLAQGC57XQBS3NT44QVQL6Y3LLC6IMJA3UBZMCD2Q, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.354304865 (588h38m55.238259118s) [noise=true , address=77.90.31.17:4804 , piece_id=FZTRYSIGA7SGLHAIV44N4A3LTVBRH3FIXY2PTT2KFOSBAYONTSTA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.456416308 (588h38m55.340370507s) [noise=true , address=145.79.227.34:4932 , piece_id=6P4HILBHZKG5PSO5UKRDZ2L2KQ3MNCGSMNTT4AKYROOM3G5XFALA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:18.064577119 (588h38m55.948531321s) [noise=true , address=64.49.13.170:4725 , piece_id=FJ7LY7HY2RKPMN6VAP5D2MNXD33XZVJUXPSBK4FWBE3MV6D7CDPQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:18.377401016 (588h38m56.261355219s) [noise=true , address=77.90.31.19:4857 , piece_id=XM2V566CPCILX4U3WMMATKJXNI52DCPN6ZHEYWN6XJLHN3SAAB3A, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:35:16.136794989 (588h39m54.020749194s) [noise=true , address=77.90.31.10:4531 , piece_id=Y7RBZXSHFOOBTISMPH6VBAMV3ZZAZAAH75NYPZP64K4DOG6J7X7A, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:1AvSrhj1nHWLBJrw1SKacnjPTi9g4AzjqKvVFoxtzAdJFYAfQt, address:77.90.31.10:4531): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.136834341 (588h39m54.020788544s) [noise=true , address=145.79.227.41:4098 , piece_id=HHLOOAW2HISRBZBNXHB3IURICY4TKH4TI7LUZ3YYHP47U5FI5EGQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:1SoFNdwfh4p5tXUkpfjQaev6tdA2ZdhsXUFvZNBsy5QrJ9zdqb, address:145.79.227.41:4098): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.137014242 (588h39m54.020968441s) [noise=true , address=145.79.227.37:4588 , piece_id=GFCOSNHBNLRETLADAMBLCNXNORCBTAHA6QIPJ6FFKRGVSCUIAP3A, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:128rq4MaAx7kfBVrh2x8WWRi8BgD7nRoJSEpJjMQvPmKgtDtVnK, address:145.79.227.37:4588): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.137049764 (588h39m54.021003964s) [noise=true , address=77.90.31.21:4897 , piece_id=NYYQ3XRMOFPQ76N7FVWX4ZMPHOLPIO55ZNYOYOMEY4AZ2FV7F3WA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:12tc8y4NueDCMnENN3Arf4ecmGAhobx4HGuCLpidZmiDFzbGu6x, address:77.90.31.21:4897): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.137189462 (588h39m54.02114367s) [noise=true , address=145.79.227.36:4522 , piece_id=H4BPWZKCOJSM32VGBOLW5OCYDVSMT2YYOPNPKX37XIN64SSBW55A, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:1zGZe6e37uop4y622fU1GbspMPVEZURS3FbMrf7je6VT9TYGFr, address:145.79.227.36:4522): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.137220651 (588h39m54.02117484s) [seg_pos={0 0}, upload=518198]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Feb 9 04:36:32.038895323 (588h41m9.922849524s) [upload=518198]: Done uploading segments...
Feb 9 04:36:32.038928075 (588h41m9.922882286s) [upload=518198]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.595862304s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [....................XX..X...................................] duration:
1.089931178s trace:
https://jaeger.dev.storj.io/trace/10c99c609f21fd50
FAIL (2026-02-09 05:00:52 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 1.089957015s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....................X..X...................................] duration:
1.183978017s trace:
https://jaeger.dev.storj.io/trace/2851863ba19d723d
FAIL (2026-02-09 05:00:51 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.184035888s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
291.636005ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/2b03593df22bf5a6
FAIL (2026-02-09 05:00:50 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout
Duration: 291.656235ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
4 out of 60 is failed (threshold=3) [....................XX..XX..................................] duration:
2.444030168s trace:
https://jaeger.dev.storj.io/trace/3417326bc88628d0
FAIL (2026-02-09 05:00:42 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.444063582s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.................X..XX.................................X....] duration:
2.836744659s trace:
https://jaeger.dev.storj.io/trace/7db938728306fbdb
FAIL (2026-02-09 05:00:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.836770281s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [.....................XXX.XX.................................] duration:
1.613601699s trace:
https://jaeger.dev.storj.io/trace/52e03ea49baff327
FAIL (2026-02-09 05:00:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.613633875s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................X.X.................................] duration:
2.266139858s trace:
https://jaeger.dev.storj.io/trace/1e478b832adc7fb5
FAIL (2026-02-09 05:00:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
Feb 9 04:34:16.425527162 (588h38m54.309481351s) [upload=518198]: Starting upload
Feb 9 04:34:16.425614910 (588h38m54.30956911s) [upload=518198]: Done starting upload
Feb 9 04:34:16.425686379 (588h38m54.309640565s) [upload=518198]: Uploading segments...
Feb 9 04:34:16.425998866 (588h38m54.309953067s) [upload=518198]: Got next segment. Inline: false
Feb 9 04:34:16.426077295 (588h38m54.310031504s) [upload=518198]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 9 04:34:16.426084296 (588h38m54.310038505s) [upload=518198]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 9 04:34:16.539403201 (588h38m54.4233574s) [seg_pos={0 0}, upload=518198]: Begin upload segment...
Feb 9 04:34:16.539636372 (588h38m54.423590581s) [noise=true , address=145.79.227.44:4200 , piece_id=THUFQP3XAQBOE6BTDSRC2BOPHJBGLXSQNQKHAG3ZQWN6XL54745Q, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.540344248 (588h38m54.424298449s) [seg_pos={0 0}, upload=518198]: Done begin upload segment.
Feb 9 04:34:16.540380789 (588h38m54.424421242s) [noise=true , address=145.79.227.37:4588 , piece_id=GFCOSNHBNLRETLADAMBLCNXNORCBTAHA6QIPJ6FFKRGVSCUIAP3A, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.540813561 (588h38m54.424767771s) [noise=true , address=77.90.31.21:4897 , piece_id=NYYQ3XRMOFPQ76N7FVWX4ZMPHOLPIO55ZNYOYOMEY4AZ2FV7F3WA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.541105928 (588h38m54.425060128s) [noise=true , address=77.90.31.18:4826 , piece_id=MUTU2BT5YTCI7SWC3UHYYK4FQE5G7PNSYDRTDPKHJV66REC5XA3Q, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.541255031 (588h38m54.425209231s) [noise=true , address=145.79.227.19:4104 , piece_id=VA6C5GNCLDKT3UWJVGXYAR3HABMQFFJ73Y57ZEVGOPVZPN3U2YZQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.541771561 (588h38m54.425725763s) [noise=true , address=79.127.219.45:4868 , piece_id=F2DZHS5VN7IFGFZAFPKNIQ7LW37JH6NNQU76IKIUBVRPRWT36TSQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.542336894 (588h38m54.426291096s) [noise=true , address=77.90.31.10:4531 , piece_id=Y7RBZXSHFOOBTISMPH6VBAMV3ZZAZAAH75NYPZP64K4DOG6J7X7A, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.542856214 (588h38m54.426810421s) [noise=true , address=64.49.13.170:4725 , piece_id=FJ7LY7HY2RKPMN6VAP5D2MNXD33XZVJUXPSBK4FWBE3MV6D7CDPQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.544116137 (588h38m54.428070342s) [noise=true , address=79.127.219.41:4931 , piece_id=TJOGRTPRIXNCIZH37FMBANDIPNZGLED2QOOPURPV7NY3E3U7LJ2Q, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.544608261 (588h38m54.428562461s) [noise=true , address=145.79.227.36:4522 , piece_id=H4BPWZKCOJSM32VGBOLW5OCYDVSMT2YYOPNPKX37XIN64SSBW55A, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.545047288 (588h38m54.429001497s) [noise=true , address=145.79.227.34:4932 , piece_id=6P4HILBHZKG5PSO5UKRDZ2L2KQ3MNCGSMNTT4AKYROOM3G5XFALA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.545675258 (588h38m54.429629466s) [noise=true , address=77.90.31.19:4857 , piece_id=XM2V566CPCILX4U3WMMATKJXNI52DCPN6ZHEYWN6XJLHN3SAAB3A, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.546288192 (588h38m54.430242394s) [noise=true , address=77.90.31.13:4653 , piece_id=733247IC2BSFCHBPRLYSOAM3YJJM73HT2K3FAHQDJA7EUO4UO2FA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.546820663 (588h38m54.430774863s) [noise=true , address=77.90.31.15:4710 , piece_id=YE25JHYBLJIMLAQGC57XQBS3NT44QVQL6Y3LLC6IMJA3UBZMCD2Q, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.547480432 (588h38m54.431434642s) [noise=true , address=145.79.227.41:4098 , piece_id=HHLOOAW2HISRBZBNXHB3IURICY4TKH4TI7LUZ3YYHP47U5FI5EGQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.548092704 (588h38m54.432046906s) [noise=true , address=79.127.219.46:4903 , piece_id=5AB6ESAUL35HINFZQ7FSVBCJYLY5QOPNF6YIQUH6YLJXZKHQ5IAQ, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.548485216 (588h38m54.432439419s) [noise=true , address=79.127.219.38:4825 , piece_id=4KWAIHWK76P2GIJDP7SNDJOLHVGJKHXI5M2J34D2JKDB2J57A3OA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:16.548835080 (588h38m54.432789285s) [noise=true , address=77.90.31.17:4804 , piece_id=FZTRYSIGA7SGLHAIV44N4A3LTVBRH3FIXY2PTT2KFOSBAYONTSTA, seg_pos={0 0}, upload=518198]: Uploading piece...
Feb 9 04:34:17.090871348 (588h38m54.974825556s) [upload=518198]: Next returned nil segment
Feb 9 04:34:17.090889807 (588h38m54.974844013s) [upload=518198]: Waiting for error group managing segments...
Feb 9 04:34:17.239393228 (588h38m55.123347429s) [noise=true , address=79.127.219.45:4868 , piece_id=F2DZHS5VN7IFGFZAFPKNIQ7LW37JH6NNQU76IKIUBVRPRWT36TSQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.253828118 (588h38m55.137782306s) [noise=true , address=79.127.219.41:4931 , piece_id=TJOGRTPRIXNCIZH37FMBANDIPNZGLED2QOOPURPV7NY3E3U7LJ2Q, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.286316140 (588h38m55.170270342s) [noise=true , address=77.90.31.13:4653 , piece_id=733247IC2BSFCHBPRLYSOAM3YJJM73HT2K3FAHQDJA7EUO4UO2FA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.287554811 (588h38m55.171509033s) [noise=true , address=77.90.31.18:4826 , piece_id=MUTU2BT5YTCI7SWC3UHYYK4FQE5G7PNSYDRTDPKHJV66REC5XA3Q, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.301101288 (588h38m55.185055487s) [noise=true , address=145.79.227.19:4104 , piece_id=VA6C5GNCLDKT3UWJVGXYAR3HABMQFFJ73Y57ZEVGOPVZPN3U2YZQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.305761625 (588h38m55.189715825s) [noise=true , address=79.127.219.46:4903 , piece_id=5AB6ESAUL35HINFZQ7FSVBCJYLY5QOPNF6YIQUH6YLJXZKHQ5IAQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.310622363 (588h38m55.194576563s) [noise=true , address=79.127.219.38:4825 , piece_id=4KWAIHWK76P2GIJDP7SNDJOLHVGJKHXI5M2J34D2JKDB2J57A3OA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.316237374 (588h38m55.200191573s) [noise=true , address=145.79.227.44:4200 , piece_id=THUFQP3XAQBOE6BTDSRC2BOPHJBGLXSQNQKHAG3ZQWN6XL54745Q, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.318039841 (588h38m55.20199404s) [noise=true , address=77.90.31.15:4710 , piece_id=YE25JHYBLJIMLAQGC57XQBS3NT44QVQL6Y3LLC6IMJA3UBZMCD2Q, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.354304865 (588h38m55.238259118s) [noise=true , address=77.90.31.17:4804 , piece_id=FZTRYSIGA7SGLHAIV44N4A3LTVBRH3FIXY2PTT2KFOSBAYONTSTA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:17.456416308 (588h38m55.340370507s) [noise=true , address=145.79.227.34:4932 , piece_id=6P4HILBHZKG5PSO5UKRDZ2L2KQ3MNCGSMNTT4AKYROOM3G5XFALA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:18.064577119 (588h38m55.948531321s) [noise=true , address=64.49.13.170:4725 , piece_id=FJ7LY7HY2RKPMN6VAP5D2MNXD33XZVJUXPSBK4FWBE3MV6D7CDPQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:34:18.377401016 (588h38m56.261355219s) [noise=true , address=77.90.31.19:4857 , piece_id=XM2V566CPCILX4U3WMMATKJXNI52DCPN6ZHEYWN6XJLHN3SAAB3A, seg_pos={0 0}, upload=518198]: Done uploading piece. err: <nil>
Feb 9 04:35:16.136794989 (588h39m54.020749194s) [noise=true , address=77.90.31.10:4531 , piece_id=Y7RBZXSHFOOBTISMPH6VBAMV3ZZAZAAH75NYPZP64K4DOG6J7X7A, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:1AvSrhj1nHWLBJrw1SKacnjPTi9g4AzjqKvVFoxtzAdJFYAfQt, address:77.90.31.10:4531): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.136834341 (588h39m54.020788544s) [noise=true , address=145.79.227.41:4098 , piece_id=HHLOOAW2HISRBZBNXHB3IURICY4TKH4TI7LUZ3YYHP47U5FI5EGQ, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:1SoFNdwfh4p5tXUkpfjQaev6tdA2ZdhsXUFvZNBsy5QrJ9zdqb, address:145.79.227.41:4098): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.137014242 (588h39m54.020968441s) [noise=true , address=145.79.227.37:4588 , piece_id=GFCOSNHBNLRETLADAMBLCNXNORCBTAHA6QIPJ6FFKRGVSCUIAP3A, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:128rq4MaAx7kfBVrh2x8WWRi8BgD7nRoJSEpJjMQvPmKgtDtVnK, address:145.79.227.37:4588): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.137049764 (588h39m54.021003964s) [noise=true , address=77.90.31.21:4897 , piece_id=NYYQ3XRMOFPQ76N7FVWX4ZMPHOLPIO55ZNYOYOMEY4AZ2FV7F3WA, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:12tc8y4NueDCMnENN3Arf4ecmGAhobx4HGuCLpidZmiDFzbGu6x, address:77.90.31.21:4897): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.137189462 (588h39m54.02114367s) [noise=true , address=145.79.227.36:4522 , piece_id=H4BPWZKCOJSM32VGBOLW5OCYDVSMT2YYOPNPKX37XIN64SSBW55A, seg_pos={0 0}, upload=518198]: Done uploading piece. err: ecclient: upload failed (node:1zGZe6e37uop4y622fU1GbspMPVEZURS3FbMrf7je6VT9TYGFr, address:145.79.227.36:4522): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:16.137220651 (588h39m54.02117484s) [seg_pos={0 0}, upload=518198]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Feb 9 04:36:32.038895323 (588h41m9.922849524s) [upload=518198]: Done uploading segments...
Feb 9 04:36:32.038928075 (588h41m9.922882286s) [upload=518198]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.266176305s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
3 out of 60 is failed (threshold=3) [.......................X.XX.................................] duration:
3.218850203s trace:
https://jaeger.dev.storj.io/trace/2545ffb6627030f4
FAIL (2026-02-09 05:00:34 UTC)
orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 3.21888142s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......................X..X.................................] duration:
1.015745048s trace:
https://jaeger.dev.storj.io/trace/7fb5679f0d1238de
FAIL (2026-02-09 05:00:19 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.015801508s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.....................X.X....................................] duration:
533.897598ms remote:
"tcp:34.2.157.232:443" target:
https://ap1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/96628cddd17f5b6
FAIL (2026-02-09 05:00:19 UTC)
orbiter warning: Get "https://ap1.storj.io/login": context deadline exceeded
Duration: 533.923011ms
core
satellite
ap1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
269.101566ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/2b03593df22bf5a6
FAIL (2026-02-09 05:00:18 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout
Duration: 269.135431ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.........................X..X...............................] duration:
3.407458445s trace:
https://jaeger.dev.storj.io/trace/46fbafc4cec1829c
FAIL (2026-02-09 05:00:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.407488916s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.....................XX..XX.................................] duration:
2.288421941s trace:
https://jaeger.dev.storj.io/trace/3417326bc88628d0
FAIL (2026-02-09 05:00:15 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.288461667s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................X..X................................] duration:
2.466664956s trace:
https://jaeger.dev.storj.io/trace/2367fd09c0ecbab9
FAIL (2026-02-09 05:00:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
upload segment...
Feb 9 04:35:22.700329253 (588h40m0.584283452s) [seg_pos={0 0}, upload=518203]: Done begin upload segment.
Feb 9 04:35:22.700460624 (588h40m0.584414826s) [noise=true , address=77.90.31.14:4664 , piece_id=NZ2S43RG6L4ISS4D5TNSLKPCOMG7NRLI5FJH5STGMZ4TSRHSCOXA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.700988853 (588h40m0.584943055s) [noise=true , address=77.90.31.21:4877 , piece_id=KSXI5U3CRHKFFRLEB6M6T4NPGNK3GQM2WWT5JFKGJRHMDTJR4SHA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.701851965 (588h40m0.585806164s) [noise=true , address=79.127.219.41:4930 , piece_id=AWPOIHF6TLMVHNHYDJKVY5OSVWT22NG33EDOHRCK23323ZMQKFGQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702127082 (588h40m0.586081284s) [noise=true , address=77.90.31.21:4880 , piece_id=C6FMFW55AYOLQDP7KLSLOJTC5G5TRXVQED7ZVEQ6NIAJJYOL7Z2Q, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702346744 (588h40m0.586300954s) [noise=true , address=45.91.116.7:4024 , piece_id=CRA3XDRATYDKNGI55D5WFKDMKZ7VSG44NHGW7N22KMEHKUNAP4QA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702509793 (588h40m0.586463991s) [noise=true , address=77.90.31.19:4844 , piece_id=6RD5ZBHA3MGQAXUUHQ3KVTYFZXMQRJMKV2DXWFEMXNS3CJEGD42A, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702814626 (588h40m0.586768829s) [noise=true , address=45.91.116.12:4228 , piece_id=DEDCYXMNUKHEIQKD2FLLJFPOTFS7QINLNFYD4SIKL42YYOSRHI4Q, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.702824399 (588h40m0.586778657s) [noise=true , address=77.90.31.14:4661 , piece_id=RMTP6QMQTSCXHDWHAHPRDNL26P5X2ZQ5NVYFNAVTELPA6THRM5AA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.703210558 (588h40m0.587164761s) [noise=true , address=77.90.31.11:4556 , piece_id=SIC2PM67TPSE6ETWMINBVBKJGJFP2I5XK6LLTAZXG6TZOFYZ33EQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.703279077 (588h40m0.587233278s) [noise=true , address=77.90.31.17:4773 , piece_id=XUD5VZGZSZAH72H4QS5FSW5BGKDWB62YDNDD3Z2VGN7EHLFJBYRQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.703549671 (588h40m0.58750387s) [noise=true , address=77.90.31.21:4878 , piece_id=NS47GXLZ2KCDRIDZOAIHBGO2U2PYS2HOBOSYW43WLISJWRCAPFNQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.703771063 (588h40m0.587725273s) [noise=true , address=79.127.219.41:4926 , piece_id=GKWSUWOKER27WJNW3MHWD6ZZATXPV4ZDC4VNBRTYOHXJMA33UBBQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.704208043 (588h40m0.588162245s) [noise=true , address=79.127.219.45:4856 , piece_id=IXWYK73XOZSITPY2PO6BBAAWVTC76CWJFFFWKXSNOGYFK6LCWQPQ, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.704629165 (588h40m0.588583365s) [noise=true , address=45.91.116.12:4244 , piece_id=N3AGJ5VS5OK6SGM6CXOELXTPI5MTEECVHFPEA2HE6HYMGEFJVI3Q, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.705058487 (588h40m0.58901269s) [noise=true , address=77.90.31.19:4842 , piece_id=DC3RQKVSIEEHNZQM4EQL54S6IHO6ORZQOITSE34LD2VJSWMR4QYA, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:22.705566550 (588h40m0.589520752s) [noise=true , address=77.90.31.15:4697 , piece_id=AQ5RTDKKEL553KLEDMNB2S7S2YMMREYA7CFNUI7M2VHGN3ISG27Q, seg_pos={0 0}, upload=518203]: Uploading piece...
Feb 9 04:35:23.857509627 (588h40m1.741463833s) [upload=518203]: Next returned nil segment
Feb 9 04:35:23.857528736 (588h40m1.741482946s) [upload=518203]: Waiting for error group managing segments...
Feb 9 04:35:24.037977395 (588h40m1.921931594s) [noise=true , address=79.127.219.41:4930 , piece_id=AWPOIHF6TLMVHNHYDJKVY5OSVWT22NG33EDOHRCK23323ZMQKFGQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:26.272463124 (588h40m4.156417324s) [noise=true , address=79.127.219.41:4926 , piece_id=GKWSUWOKER27WJNW3MHWD6ZZATXPV4ZDC4VNBRTYOHXJMA33UBBQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:34.140028924 (588h40m12.023983124s) [noise=true , address=77.90.31.15:4697 , piece_id=AQ5RTDKKEL553KLEDMNB2S7S2YMMREYA7CFNUI7M2VHGN3ISG27Q, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:37.255393413 (588h40m15.139347624s) [noise=true , address=77.90.31.14:4661 , piece_id=RMTP6QMQTSCXHDWHAHPRDNL26P5X2ZQ5NVYFNAVTELPA6THRM5AA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:41.697263539 (588h40m19.581217739s) [noise=true , address=77.90.31.19:4842 , piece_id=DC3RQKVSIEEHNZQM4EQL54S6IHO6ORZQOITSE34LD2VJSWMR4QYA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:41.901811216 (588h40m19.785765419s) [noise=true , address=45.91.116.12:4244 , piece_id=N3AGJ5VS5OK6SGM6CXOELXTPI5MTEECVHFPEA2HE6HYMGEFJVI3Q, seg_pos={0 0}, upload=518203]: Done uploading piece. err: <nil>
Feb 9 04:35:47.044981722 (588h40m24.928935927s) [noise=true , address=45.91.116.7:4024 , piece_id=CRA3XDRATYDKNGI55D5WFKDMKZ7VSG44NHGW7N22KMEHKUNAP4QA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12PTKXHL7D3hiUs3jDLnV8wBCJgrgqdeYkyghP1h4wZCMezCaR, address:45.91.116.7:4024): protocol: rpc: dial tcp 45.91.116.7:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045035145 (588h40m24.928989353s) [noise=true , address=79.127.219.45:4856 , piece_id=IXWYK73XOZSITPY2PO6BBAAWVTC76CWJFFFWKXSNOGYFK6LCWQPQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:121kT5cTDuopYFWfFYdmdj6QBRJMybyL7G2ow4mFeZFEH3QVxE6, address:79.127.219.45:4856): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:47.045265584 (588h40m24.92921979s) [noise=true , address=77.90.31.14:4664 , piece_id=NZ2S43RG6L4ISS4D5TNSLKPCOMG7NRLI5FJH5STGMZ4TSRHSCOXA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 139.59.144.22:42286->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045305927 (588h40m24.929260132s) [noise=true , address=77.90.31.21:4878 , piece_id=NS47GXLZ2KCDRIDZOAIHBGO2U2PYS2HOBOSYW43WLISJWRCAPFNQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: read tcp 139.59.144.22:37798->77.90.31.21:4878: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045384440 (588h40m24.929338638s) [noise=true , address=77.90.31.17:4773 , piece_id=XUD5VZGZSZAH72H4QS5FSW5BGKDWB62YDNDD3Z2VGN7EHLFJBYRQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 139.59.144.22:48366->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045477234 (588h40m24.929431437s) [noise=true , address=77.90.31.21:4880 , piece_id=C6FMFW55AYOLQDP7KLSLOJTC5G5TRXVQED7ZVEQ6NIAJJYOL7Z2Q, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: write tcp 139.59.144.22:44174->77.90.31.21:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045498918 (588h40m24.929453118s) [noise=true , address=77.90.31.21:4877 , piece_id=KSXI5U3CRHKFFRLEB6M6T4NPGNK3GQM2WWT5JFKGJRHMDTJR4SHA, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 139.59.144.22:38330->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045701538 (588h40m24.929655736s) [noise=true , address=77.90.31.19:4844 , piece_id=6RD5ZBHA3MGQAXUUHQ3KVTYFZXMQRJMKV2DXWFEMXNS3CJEGD42A, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:1rJArRXLFnyoegSL7XGNequq3wd1c14uzeumaR1HAkd5NdQB1t, address:77.90.31.19:4844): protocol: read tcp 139.59.144.22:42234->77.90.31.19:4844: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045832097 (588h40m24.929786297s) [noise=true , address=77.90.31.11:4556 , piece_id=SIC2PM67TPSE6ETWMINBVBKJGJFP2I5XK6LLTAZXG6TZOFYZ33EQ, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks, address:77.90.31.11:4556): protocol: read tcp 139.59.144.22:39978->77.90.31.11:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045842467 (588h40m24.929796668s) [noise=true , address=45.91.116.12:4228 , piece_id=DEDCYXMNUKHEIQKD2FLLJFPOTFS7QINLNFYD4SIKL42YYOSRHI4Q, seg_pos={0 0}, upload=518203]: Done uploading piece. err: ecclient: upload failed (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, address:45.91.116.12:4228): protocol: read tcp 139.59.144.22:55958->45.91.116.12:4228: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:47.045865196 (588h40m24.929819381s) [seg_pos={0 0}, upload=518203]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Feb 9 04:35:59.996828134 (588h40m37.88078234s) [upload=518203]: Done uploading segments...
Feb 9 04:35:59.996890408 (588h40m37.880844607s) [upload=518203]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 2.466690866s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
333.937687ms target:
https://version.storj.io/ trace:
https://jaeger.dev.storj.io/trace/65dcc1a2b145167b
FAIL (2026-02-09 05:00:14 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 333.979546ms
core
website
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
4 out of 60 is failed (threshold=3) [.......................XX..XX...............................] duration:
1.692940247s trace:
https://jaeger.dev.storj.io/trace/75241c521a7f3dcb
FAIL (2026-02-09 05:00:13 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 1.692969243s
core
uplink
ap1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [..................X..XX.................................X...] duration:
3.016710286s trace:
https://jaeger.dev.storj.io/trace/7db938728306fbdb
FAIL (2026-02-09 05:00:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.016839384s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [....................XX..X...................................] duration:
1.563768163s trace:
https://jaeger.dev.storj.io/trace/3aaeb3002c06c37e
FAIL (2026-02-09 05:00:03 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 51)
Duration: 1.563799685s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
4 out of 60 is failed (threshold=3) [......................XX..XX................................] duration:
2.231369705s trace:
https://jaeger.dev.storj.io/trace/3417326bc88628d0
FAIL (2026-02-09 04:59:58 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.231402534s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [.....................XX..X..................................] duration:
882.043366ms trace:
https://jaeger.dev.storj.io/trace/10c99c609f21fd50
FAIL (2026-02-09 04:59:57 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 882.071907ms
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [......................XXX.XX................................] duration:
1.637537056s trace:
https://jaeger.dev.storj.io/trace/52e03ea49baff327
FAIL (2026-02-09 04:59:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.63768841s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.........................X.X................................] duration:
928.858843ms trace:
https://jaeger.dev.storj.io/trace/7e5469f7055ee287
FAIL (2026-02-09 04:59:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
e...
Feb 9 04:34:35.705893606 (588h39m13.589847811s) [noise=true , address=185.167.204.206:4066 , piece_id=I6GSQDXZSHWF7POYBOQYLJU2QKDITQDFVE3P2KOBHEKSVSDKJIWA, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.707890134 (588h39m13.591844338s) [noise=true , address=89.149.201.147:4015 , piece_id=4DAM72SN2235KI22UTCVVVBRVUANAWP3D5YVQJTHNV3KB563ZI6A, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.709078016 (588h39m13.593032221s) [noise=true , address=185.167.204.206:4060 , piece_id=GME5JPQ4VVG2JII35UMC6XHEAKUWYASOJ63QFLO5NIJ7LIRL3ZWA, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.710195507 (588h39m13.594149715s) [noise=true , address=185.167.204.206:4065 , piece_id=3KFJZZ4SYCKW45PEGYY6EANCLFAZDO4DGVQCD6OBWDVTOAHKUDRQ, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.711495198 (588h39m13.5954494s) [noise=true , address=81.171.14.70:4032 , piece_id=G4SCAKGMP7M5VCXNK2BIN7SVOYOF5JCJQSUWCOIC2WR5SYYVX5SA, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.712109524 (588h39m13.596063726s) [noise=true , address=185.167.204.205:4019 , piece_id=3HIP6GSZ3Y7EEW3YBNEXSMURSNONFMAP6J3V5QLQ4JTEXY7K2F7Q, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.712699464 (588h39m13.596653666s) [noise=true , address=89.149.201.147:4016 , piece_id=IUY7UBPUW3IC5KBLOGWRTR4QOED5FN6BLCJINEHSW7SO5E7W27PA, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:35.713261874 (588h39m13.597216076s) [noise=true , address=89.149.201.147:4007 , piece_id=DQLCJPMI5U6MFTBTTUZAADXHFOGI4RNDYOSV6VOPPIJ2ZNQIXHQQ, seg_pos={0 0}, upload=518201]: Uploading piece...
Feb 9 04:34:36.430571544 (588h39m14.314525751s) [upload=518201]: Next returned nil segment
Feb 9 04:34:36.430612826 (588h39m14.314567032s) [upload=518201]: Waiting for error group managing segments...
Feb 9 04:34:37.285542315 (588h39m15.169496516s) [noise=true , address=89.149.201.147:4020 , piece_id=AT4BOPLHXF6T72VGDVV3MN7CSW6HXDI4MHPKDMHHQYDVTRY2XD7Q, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:37.352144984 (588h39m15.23609919s) [noise=true , address=89.149.201.147:4011 , piece_id=JUOZTOX2S7JFJDPU632FXSQUGPW3NIKYOYKB7DFAZHHARFQDJKWQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:37.474326245 (588h39m15.358280455s) [noise=true , address=81.171.14.70:4046 , piece_id=CYD2R5P2GCFCKXOXG5TPQFU7SB66JN24CWQKTXAL2EMMZTMEJNNA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:37.706787293 (588h39m15.590741558s) [noise=true , address=81.171.14.70:4035 , piece_id=LYHOYHAN6CJOXUK7PUL7AJGQDOQMD234FY7LO3TKF7HCQK345NCQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:37.974945056 (588h39m15.858899257s) [noise=true , address=185.167.204.206:4057 , piece_id=NQ2DR2IDYK5BR77YUC7G2A2YBVMXJ6CXZ2GQIP5332RSPHUOT3SQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:38.080441216 (588h39m15.964395418s) [noise=true , address=81.171.14.70:4045 , piece_id=G35PNKGJFVLFMJ4OUHIEJTXLHD35AYSSXCNNLIVMJPAV2ZVL4HGQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:38.486602188 (588h39m16.370556388s) [noise=true , address=185.167.204.206:4061 , piece_id=OT2TOUL5ARMB476CHGPNP6NXZIUXJRHS4MTKS24LH2XYYE5CTUHQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:38.740717181 (588h39m16.62467138s) [noise=true , address=185.167.204.206:4047 , piece_id=UH4Y5DREYE4ROEIW77ZSQHFBCHWBEFMBJ4KI27MQV4LWZI47MPLQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:38.773718899 (588h39m16.657673102s) [noise=true , address=89.149.201.147:4007 , piece_id=DQLCJPMI5U6MFTBTTUZAADXHFOGI4RNDYOSV6VOPPIJ2ZNQIXHQQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:38.966510172 (588h39m16.850464373s) [noise=true , address=185.167.204.205:4012 , piece_id=EOCCFH625F326UVQXAFOH354JRJBLB74ZZDCBONQAIINRJG6UW4A, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:39.816146031 (588h39m17.700100235s) [noise=true , address=81.171.14.70:4040 , piece_id=PTNV3ZZLLZLQNUHVTOD654PRBEMFLV6T5G3ULMZW24NBXKMFKZVA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:41.016111070 (588h39m18.900065269s) [noise=true , address=81.171.14.70:4034 , piece_id=FDKTEAXHBMFPXNLZCAB4NDXXKVUMHMFRX6QXBLJQ3MROXVINDPPQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:42.460071941 (588h39m20.344026131s) [noise=true , address=81.171.14.70:4044 , piece_id=N4J4UTUGGXUNQCMXIMNS6HV7XXGO3OPS6HFCMPY5SVK2HFNBXR4Q, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:48.752914118 (588h39m26.636868321s) [noise=true , address=185.167.204.206:4060 , piece_id=GME5JPQ4VVG2JII35UMC6XHEAKUWYASOJ63QFLO5NIJ7LIRL3ZWA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:58.087687791 (588h39m35.971641979s) [noise=true , address=185.167.204.206:4065 , piece_id=3KFJZZ4SYCKW45PEGYY6EANCLFAZDO4DGVQCD6OBWDVTOAHKUDRQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:34:59.132504306 (588h39m37.016458494s) [noise=true , address=81.171.14.70:4041 , piece_id=CODI7DQA7CC66YPS56IUNXMFCFHEXX4SYCEEYGF4TYXXQFEM7WHA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:35:01.532538593 (588h39m39.4164928s) [noise=true , address=89.149.201.147:4016 , piece_id=IUY7UBPUW3IC5KBLOGWRTR4QOED5FN6BLCJINEHSW7SO5E7W27PA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:35:03.766981205 (588h39m41.650935408s) [noise=true , address=89.149.201.147:4015 , piece_id=4DAM72SN2235KI22UTCVVVBRVUANAWP3D5YVQJTHNV3KB563ZI6A, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:35:09.427240351 (588h39m47.311194536s) [noise=true , address=89.149.201.147:4013 , piece_id=6OUUA7ZOINHMDPGRJ44HZDTEJS522BTSBM3F42IESIL734DEHUUA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:35:14.481090345 (588h39m52.365044556s) [noise=true , address=185.167.204.205:4026 , piece_id=VGMKJEZ65M7SUDO7BHITUSAKOQCIQJGI3MNV6XLTIFG7X3JPDK2Q, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:35:16.756347933 (588h39m54.640302133s) [noise=true , address=81.171.14.70:4028 , piece_id=AMO232UCVQTFA6O62VUWDXVGODDHPQRFMCGRAMNGARZHCGEARQDA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: <nil>
Feb 9 04:35:34.283098105 (588h40m12.167052314s) [noise=true , address=185.167.204.205:4003 , piece_id=UZN6VHXQ4CMZ3TOKA4OPTGONEYNMAKPW6Z3W32D7XJTN625JDZ7A, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:1CjWvvJ4FdZc41sRgsQFWz4Y1ATZKgzsvDcUTmBSrCC15ekCmF, address:185.167.204.205:4003): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:34.283115609 (588h40m12.167069803s) [noise=true , address=81.171.14.70:4032 , piece_id=G4SCAKGMP7M5VCXNK2BIN7SVOYOF5JCJQSUWCOIC2WR5SYYVX5SA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 139.59.144.22:52900->81.171.14.70:4032: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:34.283347033 (588h40m12.167301242s) [noise=true , address=185.167.204.206:4069 , piece_id=LGOOCFX4YC53O5DH6IUFYXKFNJNZTM7BFLA7KLU3HYKARMLZ2WOA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:34.283388338 (588h40m12.167342536s) [noise=true , address=89.149.201.147:4009 , piece_id=4P2UU2WFURMXZQFYKEUTDD3NGXSFGGPQ3ACEJCYGCYXHHGDERKNA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:1sbZWJFMVsppCpbFVfn33bx5uGkhtFpzJm8a2i8DoAfQNsWY5F, address:89.149.201.147:4009): protocol: expected piece hash; context deadline exceeded; EOF
Feb 9 04:35:34.283661254 (588h40m12.167615456s) [noise=true , address=89.149.201.147:4014 , piece_id=D6EFC33PCPWUACSSEO4INMWDG557CZYQYFAAOI3JFFPBPLRFXNTQ, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 139.59.144.22:53384->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:34.284010218 (588h40m12.167964417s) [noise=true , address=185.167.204.205:4019 , piece_id=3HIP6GSZ3Y7EEW3YBNEXSMURSNONFMAP6J3V5QLQ4JTEXY7K2F7Q, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:12CwWyCCjbNNuuSxpk9PhiWxG2yvuT19gDyJhezMfUEez6LrpDj, address:185.167.204.205:4019): protocol: read tcp 139.59.144.22:58778->185.167.204.205:4019: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:34.284176217 (588h40m12.168130427s) [noise=true , address=185.167.204.206:4066 , piece_id=I6GSQDXZSHWF7POYBOQYLJU2QKDITQDFVE3P2KOBHEKSVSDKJIWA, seg_pos={0 0}, upload=518201]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: read tcp 139.59.144.22:38920->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 9 04:35:34.284215153 (588h40m12.168169349s) [seg_pos={0 0}, upload=518201]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Feb 9 04:35:34.334736863 (588h40m12.218691063s) [upload=518201]: Done uploading segments...
Feb 9 04:35:34.334794137 (588h40m12.218748348s) [upload=518201]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 928.889525ms
core
uplink
eu-select-1
eu1-placement-15
|