Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XXXXXXXXXXXXX.....................................XXXXXXXXXX] duration: 30.000459313s FAIL (2025-08-07 22:20:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000476534s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 24 out of 60 is failed (threshold=3) [XXXXXXXXXXXXX....................................XXXXXXXXXXX] duration: 30.000830462s FAIL (2025-08-07 22:20:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== (12h32m40.92738489s) [upload=12933]: Uploading segments... Aug 7 22:12:19.183400189 (12h32m40.92745706s) [upload=12933]: Got next segment. Inline: true Aug 7 22:12:19.183407329 (12h32m40.927464177s) [upload=12933]: Waiting for error group managing segments... Aug 7 22:12:19.183479311 (12h32m40.927536158s) [upload=12933]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:12:19.183484643 (12h32m40.927541506s) [upload=12933]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:12:19.183487379 (12h32m40.927544225s) [upload=12933]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:12:48.909423390 (12h33m10.653480257s) [upload=12933]: Done uploading segments... Aug 7 22:12:48.909444894 (12h33m10.653501768s) [upload=12933]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:13:10.831892676 (12h33m32.575949525s) [upload=12948]: Starting upload Aug 7 22:13:10.831993207 (12h33m32.576050068s) [upload=12948]: Done starting upload Aug 7 22:13:10.832071605 (12h33m32.576128457s) [upload=12948]: Uploading segments... Aug 7 22:13:10.832146431 (12h33m32.576203291s) [upload=12948]: Got next segment. Inline: true Aug 7 22:13:10.832152814 (12h33m32.576209674s) [upload=12948]: Waiting for error group managing segments... Aug 7 22:13:10.832208966 (12h33m32.576265827s) [upload=12948]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:13:10.832214384 (12h33m32.576271244s) [upload=12948]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:13:10.832217904 (12h33m32.576274764s) [upload=12948]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:13:40.559928781 (12h34m2.303985629s) [upload=12948]: Done uploading segments... Aug 7 22:13:40.559950271 (12h34m2.304007117s) [upload=12948]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:13:48.685772427 (12h34m10.429829287s) [upload=12958]: Starting upload Aug 7 22:13:48.685858893 (12h34m10.429915753s) [upload=12958]: Done starting upload Aug 7 22:13:48.685982193 (12h34m10.430039042s) [upload=12958]: Uploading segments... Aug 7 22:13:48.686047966 (12h34m10.430104835s) [upload=12958]: Got next segment. Inline: true Aug 7 22:13:48.686053563 (12h34m10.430110423s) [upload=12958]: Waiting for error group managing segments... Aug 7 22:13:48.686096300 (12h34m10.43015316s) [upload=12958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:13:48.686101341 (12h34m10.430158201s) [upload=12958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:13:48.686104243 (12h34m10.430161104s) [upload=12958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:14:18.399606594 (12h34m40.143663447s) [upload=12958]: Done uploading segments... Aug 7 22:14:18.399627822 (12h34m40.143684675s) [upload=12958]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:14:59.902619970 (12h35m21.646676816s) [upload=12977]: Starting upload Aug 7 22:14:59.902685895 (12h35m21.646742756s) [upload=12977]: Done starting upload Aug 7 22:14:59.902751694 (12h35m21.646808545s) [upload=12977]: Uploading segments... Aug 7 22:14:59.902795346 (12h35m21.646852214s) [upload=12977]: Got next segment. Inline: true Aug 7 22:14:59.902801495 (12h35m21.646858346s) [upload=12977]: Waiting for error group managing segments... Aug 7 22:14:59.902858709 (12h35m21.64691556s) [upload=12977]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:14:59.902863962 (12h35m21.646920811s) [upload=12977]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:14:59.902867458 (12h35m21.646924309s) [upload=12977]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:15:29.636302762 (12h35m51.380359612s) [upload=12977]: Done uploading segments... Aug 7 22:15:29.636325398 (12h35m51.380382249s) [upload=12977]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:45.428850713 (12h36m7.172907563s) [upload=12988]: Starting upload Aug 7 22:15:45.428978829 (12h36m7.173035696s) [upload=12988]: Done starting upload Aug 7 22:15:45.429103446 (12h36m7.173160298s) [upload=12988]: Uploading segments... Aug 7 22:15:45.429150754 (12h36m7.173207623s) [upload=12988]: Got next segment. Inline: true Aug 7 22:15:45.429156339 (12h36m7.173213199s) [upload=12988]: Waiting for error group managing segments... Aug 7 22:15:45.429230361 (12h36m7.173287221s) [upload=12988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:15:45.429247463 (12h36m7.173304325s) [upload=12988]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:15:45.429251133 (12h36m7.173307993s) [upload=12988]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:16:15.155440875 (12h36m36.899497725s) [upload=12988]: Done uploading segments... Aug 7 22:16:15.155471506 (12h36m36.899528367s) [upload=12988]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:16:15.598565945 (12h36m37.342622791s) [upload=12999]: Starting upload Aug 7 22:16:15.598645050 (12h36m37.342701913s) [upload=12999]: Done starting upload Aug 7 22:16:15.598718650 (12h36m37.342775497s) [upload=12999]: Uploading segments... Aug 7 22:16:15.598774264 (12h36m37.34283113s) [upload=12999]: Got next segment. Inline: true Aug 7 22:16:15.598779124 (12h36m37.342835984s) [upload=12999]: Waiting for error group managing segments... Aug 7 22:16:15.598830461 (12h36m37.342887324s) [upload=12999]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:16:15.598854102 (12h36m37.342910963s) [upload=12999]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:16:15.598857833 (12h36m37.342914694s) [upload=12999]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:16:45.328584439 (12h37m7.072641289s) [upload=12999]: Done uploading segments... Aug 7 22:16:45.328609005 (12h37m7.072665856s) [upload=12999]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:27.480756310 (12h37m49.22481316s) [upload=13015]: Starting upload Aug 7 22:17:27.480859326 (12h37m49.22491619s) [upload=13015]: Done starting upload Aug 7 22:17:27.480944450 (12h37m49.225001301s) [upload=13015]: Uploading segments... Aug 7 22:17:27.481000891 (12h37m49.225057757s) [upload=13015]: Got next segment. Inline: true Aug 7 22:17:27.481006468 (12h37m49.225063328s) [upload=13015]: Waiting for error group managing segments... Aug 7 22:17:27.481069727 (12h37m49.225126586s) [upload=13015]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:17:27.481076622 (12h37m49.225133484s) [upload=13015]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:27.481081288 (12h37m49.225138155s) [upload=13015]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:17:57.211870572 (12h38m18.955927423s) [upload=13015]: Done uploading segments... Aug 7 22:17:57.211892595 (12h38m18.955949444s) [upload=13015]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:14.331617780 (12h38m36.07567463s) [upload=13026]: Starting upload Aug 7 22:18:14.331690520 (12h38m36.07574737s) [upload=13026]: Done starting upload Aug 7 22:18:14.331789874 (12h38m36.075846724s) [upload=13026]: Uploading segments... Aug 7 22:18:14.331829497 (12h38m36.075886363s) [upload=13026]: Got next segment. Inline: true Aug 7 22:18:14.331834588 (12h38m36.075891448s) [upload=13026]: Waiting for error group managing segments... Aug 7 22:18:14.331890028 (12h38m36.075946878s) [upload=13026]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:18:14.331895594 (12h38m36.075952444s) [upload=13026]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:14.331899378 (12h38m36.075956238s) [upload=13026]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:18:44.050649404 (12h39m5.794706254s) [upload=13026]: Done uploading segments... Aug 7 22:18:44.050671176 (12h39m5.794728026s) [upload=13026]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:08.682705115 (12h39m30.42676198s) [upload=13039]: Starting upload Aug 7 22:19:08.682797967 (12h39m30.426854836s) [upload=13039]: Done starting upload Aug 7 22:19:08.682924116 (12h39m30.426980968s) [upload=13039]: Uploading segments... Aug 7 22:19:08.682986486 (12h39m30.427043358s) [upload=13039]: Got next segment. Inline: true Aug 7 22:19:08.682993964 (12h39m30.427050856s) [upload=13039]: Waiting for error group managing segments... Aug 7 22:19:08.683054026 (12h39m30.427110894s) [upload=13039]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:19:08.683060373 (12h39m30.427117244s) [upload=13039]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:08.683064440 (12h39m30.42712131s) [upload=13039]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:19:38.406368834 (12h40m0.150425697s) [upload=13039]: Done uploading segments... Aug 7 22:19:38.406426692 (12h40m0.150483554s) [upload=13039]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:20.517418397 (12h40m42.26147525s) [upload=13053]: Starting upload Aug 7 22:20:20.517575991 (12h40m42.261632856s) [upload=13053]: Done starting upload Aug 7 22:20:20.517715190 (12h40m42.261772059s) [upload=13053]: Uploading segments... Aug 7 22:20:20.517901368 (12h40m42.26195823s) [upload=13053]: Got next segment. Inline: true Aug 7 22:20:20.517963476 (12h40m42.262020457s) [upload=13053]: Waiting for error group managing segments... Aug 7 22:20:20.518079988 (12h40m42.262137053s) [upload=13053]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:20:20.518123882 (12h40m42.262180743s) [upload=13053]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:20.518130819 (12h40m42.262187685s) [upload=13053]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:20:50.250392650 (12h41m11.994449511s) [upload=13053]: Done uploading segments... Aug 7 22:20:50.250418123 (12h41m11.994474984s) [upload=13053]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000855344s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 26 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXX......X.X...........................XXXXXXXXX] duration: 1m0.000490554s FAIL (2025-08-07 22:20:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000514127s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [X.X..................................................X......] duration: 288.568693ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:20:27 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 288.641989ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 17 out of 60 is failed (threshold=3) [XXXXXXX.X..........................................XXXXXXXXX] duration: 30.00061999s FAIL (2025-08-07 22:20:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00064596s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXX.....................................XXXXXXXXX] duration: 30.000916729s FAIL (2025-08-07 22:20:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000945776s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 10 out of 60 is failed (threshold=3) [....................XXXX..............................XXXXXX] duration: 1m0.001376224s FAIL (2025-08-07 22:20:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 57.134173791 (12h32m18.878230638s) [noise=true , address=104.36.82.13:4188 , piece_id=C24Q324TD3OIGJSBOKVUN2VH5ORNKINWDHBTA3O56VSRPJN52SGA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.151552906 (12h32m18.895609777s) [noise=true , address=104.36.82.6:4895 , piece_id=HUPRXOSI2CS2UDOF47BME2BYCCLXANUCJAYVZV4JNRMPQQUDZQFA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.151985548 (12h32m18.896042414s) [noise=true , address=104.36.82.60:4343 , piece_id=A5ZQY7FYEN7BVIGGB6T43UTFNOC6NGPBM3XJ23QSLVRFFVSVYJCQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.177078648 (12h32m18.921135515s) [noise=true , address=104.36.82.34:4566 , piece_id=GR2IDGJL2TB5VYWZOIW2C652I5B5XUUXXI4SIELMP3V4GMO2ZH4Q, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.180968885 (12h32m18.925025749s) [noise=true , address=104.36.82.45:4940 , piece_id=E2BSUMWRGNFOCNBBT32GAMW3JN4PFKNKSYXK5AEUGQQKRK4I6FKA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.182573730 (12h32m18.926630601s) [noise=true , address=104.36.82.43:4837 , piece_id=WALP4A6IL6VR6577XHAIATXZGBECNO6BL5Y5Z4I4DMPCNIK7X5QQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.185217873 (12h32m18.929274727s) [noise=true , address=104.36.82.60:4346 , piece_id=OCI6XALWGEOAF64WRNURL5VVPFWYBT24GXK4GNQMUX2MJYZACROQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.191081154 (12h32m18.935138022s) [noise=true , address=104.36.82.23:4448 , piece_id=RD4YTZPX2A4SRE2XWYS45DWZRG6CUS5VLCYBMJPBJ24LGJTGITFQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.205956300 (12h32m18.950013175s) [noise=true , address=104.36.82.36:4645 , piece_id=P63BF2T4FLUK7DMM4GQY277QTDGZXT3CD4Z5GFDMUQTTJN77TZIQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.214668342 (12h32m18.958725212s) [noise=true , address=104.36.82.39:4733 , piece_id=VZPW75CLRYOWCVQGWBWAOGWKFCXYLYOM6OIDMXRBR5IRGZ7DGLLQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.245950582 (12h32m18.990007453s) [noise=true , address=104.36.82.16:4310 , piece_id=UAVZCLFOGCC6T4HX676IX6OWFNRZRTXL3PITG5IZFR2JATIZS2DQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.280359350 (12h32m19.024416213s) [noise=true , address=45.91.116.9:4272 , piece_id=3QGHJZTVECLQTJGC46GZ5276LVTHR36LBTK5Q5FSRGD7QXOUJJ6A, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.338710901 (12h32m19.082767752s) [noise=true , address=45.91.116.13:4166 , piece_id=HSFXWLME6GIFAWHD73GW4H6CBPBJZ4MPTB7JD7XY2YBTZESICNRA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.617684582 (12h32m19.361741428s) [noise=true , address=104.36.82.62:4439 , piece_id=2P7NHL5WLDREVBGDS4IMRS6D4DZBQHRYFM7USCDK2KU74SKDLW4A, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.617723267 (12h32m19.361780128s) [seg_pos={0 0}, upload=12925]: Segment reached optimal threshold of 14 pieces. Aug 7 22:11:57.617992734 (12h32m19.36204958s) [noise=true , address=104.36.82.47:4520 , piece_id=DLEDG3JMSGGGFQFGNJWV4XIDPXITG54XCYTRXXJIRR4DOXWIDAVA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 22:11:57.618089635 (12h32m19.362146495s) [noise=true , address=64.49.13.137:4498 , piece_id=7SLUBM4U2CKK25FCFPHVKBA4D4TKCRZXKASC7LR2ASBM7C7MDQKQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: expected piece hash; context canceled; EOF Aug 7 22:11:57.618135706 (12h32m19.362192556s) [noise=true , address=104.36.82.5:4855 , piece_id=OEDGYJGQIIJHLEBR5OPPQCO75KXIGPFZBDCCTBYSUV7XQADJRSAA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh): protocol: write tcp 139.59.144.22:57090->104.36.82.5:4855: use of closed network connection; write tcp 139.59.144.22:57090->104.36.82.5:4855: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57090->104.36.82.5:4855: use of closed network connection Aug 7 22:11:57.618165938 (12h32m19.362222785s) [noise=true , address=64.49.13.139:4471 , piece_id=QTORROJDLNZQTDC2VFU3Q5DW47TA7VM7FZAVKTACLLCPMHGOC52A, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129YkXH84aYSubinAC57swW24qCCEtvJTSMDGoxP9WdgpGcZxLr): protocol: expected piece hash; context canceled; EOF Aug 7 22:11:57.618181168 (12h32m19.362238028s) [seg_pos={0 0}, upload=12925]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:11:57.618262981 (12h32m19.362319833s) [upload=12925]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:11:57.618268565 (12h32m19.362325425s) [upload=12925]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:15:11.462898274 (12h35m33.206955137s) [upload=12925]: Done uploading segments... Aug 7 22:15:11.462927061 (12h35m33.206983911s) [upload=12925]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:11.893865947 (12h35m33.637922798s) [upload=12978]: Starting upload Aug 7 22:15:11.894006161 (12h35m33.638063025s) [upload=12978]: Done starting upload Aug 7 22:15:11.894128456 (12h35m33.63818531s) [upload=12978]: Uploading segments... Aug 7 22:15:11.894439225 (12h35m33.638496094s) [upload=12978]: Got next segment. Inline: false Aug 7 22:15:11.894516777 (12h35m33.638573639s) [upload=12978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:15:11.894545833 (12h35m33.638602694s) [upload=12978]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:16:11.626077800 (12h36m33.370134666s) [upload=12978]: Next segment err: context deadline exceeded Aug 7 22:16:11.626165441 (12h36m33.370222301s) [upload=12978]: Done uploading segments... Aug 7 22:16:11.626183066 (12h36m33.370239913s) [upload=12978]: Upload finished. err: context deadline exceeded Aug 7 22:16:12.058909015 (12h36m33.802965875s) [upload=12997]: Starting upload Aug 7 22:16:12.058982372 (12h36m33.803039232s) [upload=12997]: Done starting upload Aug 7 22:16:12.059120653 (12h36m33.803177509s) [upload=12997]: Uploading segments... Aug 7 22:16:12.059593126 (12h36m33.803649985s) [upload=12997]: Got next segment. Inline: false Aug 7 22:16:12.059677172 (12h36m33.803734023s) [upload=12997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:16:12.059684424 (12h36m33.803741287s) [upload=12997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:17:11.787443939 (12h37m33.531500809s) [upload=12997]: Next segment err: context deadline exceeded Aug 7 22:17:11.787498695 (12h37m33.531555556s) [upload=12997]: Done uploading segments... Aug 7 22:17:11.787527052 (12h37m33.531583904s) [upload=12997]: Upload finished. err: context deadline exceeded Aug 7 22:17:12.226578214 (12h37m33.970635065s) [upload=13013]: Starting upload Aug 7 22:17:12.226666960 (12h37m33.970723825s) [upload=13013]: Done starting upload Aug 7 22:17:12.226793229 (12h37m33.970850079s) [upload=13013]: Uploading segments... Aug 7 22:17:12.227044345 (12h37m33.971101196s) [upload=13013]: Got next segment. Inline: false Aug 7 22:17:12.227107713 (12h37m33.971164563s) [upload=13013]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:17:12.227113591 (12h37m33.971170444s) [upload=13013]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:18:11.956251664 (12h38m33.700308534s) [upload=13013]: Next segment err: context deadline exceeded Aug 7 22:18:11.956360027 (12h38m33.700416877s) [upload=13013]: Done uploading segments... Aug 7 22:18:11.956379323 (12h38m33.700436185s) [upload=13013]: Upload finished. err: context deadline exceeded Aug 7 22:18:12.371524537 (12h38m34.115581388s) [upload=13024]: Starting upload Aug 7 22:18:12.371599713 (12h38m34.115656573s) [upload=13024]: Done starting upload Aug 7 22:18:12.371686548 (12h38m34.115743409s) [upload=13024]: Uploading segments... Aug 7 22:18:12.371885224 (12h38m34.115942074s) [upload=13024]: Got next segment. Inline: false Aug 7 22:18:12.371926111 (12h38m34.115982962s) [upload=13024]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:18:12.371931665 (12h38m34.115988525s) [upload=13024]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:19:12.105017725 (12h39m33.849074594s) [upload=13024]: Next segment err: context deadline exceeded Aug 7 22:19:12.105111880 (12h39m33.849168731s) [upload=13024]: Done uploading segments... Aug 7 22:19:12.105162034 (12h39m33.849218885s) [upload=13024]: Upload finished. err: context deadline exceeded Aug 7 22:19:14.319119032 (12h39m36.063175896s) [upload=13040]: Starting upload Aug 7 22:19:14.319296351 (12h39m36.063353203s) [upload=13040]: Done starting upload Aug 7 22:19:14.319413921 (12h39m36.063470785s) [upload=13040]: Uploading segments... Aug 7 22:19:14.319749644 (12h39m36.063806505s) [upload=13040]: Got next segment. Inline: false Aug 7 22:19:14.319821004 (12h39m36.063877869s) [upload=13040]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:19:14.319827454 (12h39m36.063884319s) [upload=13040]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:20:14.045204095 (12h40m35.789261177s) [upload=13040]: Next segment err: context deadline exceeded Aug 7 22:20:14.045278008 (12h40m35.789334861s) [upload=13040]: Done uploading segments... Aug 7 22:20:14.045305281 (12h40m35.789362135s) [upload=13040]: Upload finished. err: context deadline exceeded Duration: 1m0.001415547s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 22 out of 60 is failed (threshold=3) [XXXXXXX..X.XX..................................X.XXXXXXXXXXX] duration: 30.000916923s FAIL (2025-08-07 22:20:07 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000934655s core uplink us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 123.572588ms remote: "tcp:185.244.226.2:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-07 22:20:03 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 123.628546ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X.....X.........X..X...........X..................] duration: 3.678288319s FAIL (2025-08-07 22:19:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 5.727759364s) [upload=12820]: Starting upload Aug 7 22:05:23.983754833 (12h25m45.727811679s) [upload=12820]: Done starting upload Aug 7 22:05:23.983988533 (12h25m45.728045379s) [upload=12820]: Uploading segments... Aug 7 22:05:23.984022348 (12h25m45.728079214s) [upload=12820]: Got next segment. Inline: false Aug 7 22:05:23.984055700 (12h25m45.728112548s) [upload=12820]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:23.984060538 (12h25m45.728117398s) [upload=12820]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:05:24.185392691 (12h25m45.929449561s) [seg_pos={0 0}, upload=12820]: Begin upload segment... Aug 7 22:05:24.185557956 (12h25m45.929614817s) [seg_pos={0 0}, upload=12820]: Done begin upload segment. Aug 7 22:05:24.185603301 (12h25m45.929660177s) [noise=true , address=64.49.13.169:4705 , piece_id=6C45BITH3QWBJ22PLPILYA6LOUQ2JF2D2LOAL2BVJQQWXSXQ37IQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186037040 (12h25m45.930093906s) [noise=true , address=104.36.82.12:4174 , piece_id=DCAYU4CKMHLHOHAN7MKLV5KJTRCCC2IOWSPFPWEMAU3FSIVIXRQA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186570414 (12h25m45.930627287s) [noise=true , address=104.36.82.16:4314 , piece_id=KX4ZY5LSNB6YBYAS4NKW2QQXXQZV42VGR2TRRKZIL6ZIO3M7BAQQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186824559 (12h25m45.930881425s) [noise=true , address=104.36.82.46:4496 , piece_id=XNRPMJSQ3BNIILAQDMRYLFM623WHRZ63ND4RDTHJYLSWWOPNJGHA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187208122 (12h25m45.931264991s) [noise=true , address=104.36.82.43:4868 , piece_id=QSDRGPBV3KWFHHKWJWKLJ3N5APRHOI66TTWAWNOJDCRXQU2P7PAA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187658761 (12h25m45.931715627s) [noise=true , address=104.36.82.60:4346 , piece_id=CJPTIG7EI264SA2JCH5326GTL33YW6ASGJPXO5ZRWRDD3L5BQTYA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187940002 (12h25m45.931996872s) [noise=true , address=64.49.13.140:4515 , piece_id=WQSSOLYXPIBGHNVEG5IWDPLM45QDWPATVU73OJJTRU7NJRQCHDKA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188116203 (12h25m45.932173069s) [noise=true , address=45.91.116.6:4982 , piece_id=LAWUMX2HZNNHRQ3FUODV65COBDIMOZN2HBCSPD42P5DEC4N4VABQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188531115 (12h25m45.932587976s) [noise=true , address=104.36.82.58:4292 , piece_id=RJJS3MVCC3B3RTAXEM3XLHRSWZEUKIJUSILSKJKT5LJCWNEHX5GA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188637273 (12h25m45.932694145s) [noise=true , address=104.36.82.5:4862 , piece_id=W6AHEAV5E5RQR3YQ3LFOO33URBFGYOVTT23S5JKPP7MDNIRFHV5Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188961006 (12h25m45.933017872s) [noise=true , address=104.36.82.22:4530 , piece_id=U33JUCE4QFPT54RRVY57EF64QS7C4TUO7MWOOGZFAIHBZQUZFB3A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189183712 (12h25m45.933240578s) [noise=true , address=64.49.13.26:4945 , piece_id=BODENY6IN5IS2I2RJ3J3EV3XYKZD4B4RR4PQNKSEXYXVUVYPMC6A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189352560 (12h25m45.933409422s) [noise=true , address=104.36.82.56:4233 , piece_id=ING27ZVSXOL6ASCWIVGSF2KESQ4N6O4AXGQ3HF5EZK5TZ5T3SNNQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189640677 (12h25m45.93369755s) [noise=true , address=104.36.82.20:4461 , piece_id=XGB6IA7NPLFF7XKYATSOG54O7U36IX6BVHVWGA53T7MNVBFM5R2A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189855650 (12h25m45.933912517s) [noise=true , address=104.36.82.62:4439 , piece_id=TQYPKEGJFIZ7D26GWFTFSU4PU75J5DX24QKUEEUUX66EINYJEMIQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190187162 (12h25m45.934244033s) [noise=true , address=104.36.82.13:4201 , piece_id=PD7C77UABKPFT2WM7BOTNOGQV22DUPGWWOGZ5KUUMUVMWSSSHTJA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190334028 (12h25m45.934390898s) [noise=true , address=104.36.82.47:4534 , piece_id=JS7S2OPLDOU5C52ZACTR77R2OVC56PBN6BDNGAPRBNN3UHXQPX3Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190661949 (12h25m45.934718816s) [noise=true , address=77.90.31.16:4742 , piece_id=5USKIHBGUAV6G3BX5BHC6LOEEHDNAJUFK45RWH6OC25QGOMH7V4Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.754845434 (12h25m46.498902284s) [upload=12820]: Next returned nil segment Aug 7 22:05:24.754862619 (12h25m46.49891948s) [upload=12820]: Waiting for error group managing segments... Aug 7 22:05:24.937656798 (12h25m46.681713649s) [noise=true , address=45.91.116.6:4982 , piece_id=LAWUMX2HZNNHRQ3FUODV65COBDIMOZN2HBCSPD42P5DEC4N4VABQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:24.980443483 (12h25m46.724500333s) [noise=true , address=104.36.82.12:4174 , piece_id=DCAYU4CKMHLHOHAN7MKLV5KJTRCCC2IOWSPFPWEMAU3FSIVIXRQA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:24.995491860 (12h25m46.739551452s) [noise=true , address=77.90.31.16:4742 , piece_id=5USKIHBGUAV6G3BX5BHC6LOEEHDNAJUFK45RWH6OC25QGOMH7V4Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.083772095 (12h25m46.827828964s) [noise=true , address=104.36.82.46:4496 , piece_id=XNRPMJSQ3BNIILAQDMRYLFM623WHRZ63ND4RDTHJYLSWWOPNJGHA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.147179204 (12h25m46.891236066s) [noise=true , address=104.36.82.13:4201 , piece_id=PD7C77UABKPFT2WM7BOTNOGQV22DUPGWWOGZ5KUUMUVMWSSSHTJA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.274393636 (12h25m47.018450492s) [noise=true , address=104.36.82.62:4439 , piece_id=TQYPKEGJFIZ7D26GWFTFSU4PU75J5DX24QKUEEUUX66EINYJEMIQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.452784464 (12h25m47.196841314s) [noise=true , address=64.49.13.169:4705 , piece_id=6C45BITH3QWBJ22PLPILYA6LOUQ2JF2D2LOAL2BVJQQWXSXQ37IQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.474459740 (12h25m47.218516586s) [noise=true , address=64.49.13.26:4945 , piece_id=BODENY6IN5IS2I2RJ3J3EV3XYKZD4B4RR4PQNKSEXYXVUVYPMC6A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.481247102 (12h25m47.225303949s) [noise=true , address=64.49.13.140:4515 , piece_id=WQSSOLYXPIBGHNVEG5IWDPLM45QDWPATVU73OJJTRU7NJRQCHDKA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.666066275 (12h25m47.410123138s) [noise=true , address=104.36.82.58:4292 , piece_id=RJJS3MVCC3B3RTAXEM3XLHRSWZEUKIJUSILSKJKT5LJCWNEHX5GA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.666429351 (12h25m47.410486202s) [noise=true , address=104.36.82.56:4233 , piece_id=ING27ZVSXOL6ASCWIVGSF2KESQ4N6O4AXGQ3HF5EZK5TZ5T3SNNQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.782101005 (12h25m47.526157853s) [noise=true , address=104.36.82.16:4314 , piece_id=KX4ZY5LSNB6YBYAS4NKW2QQXXQZV42VGR2TRRKZIL6ZIO3M7BAQQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.863469768 (12h25m47.607526622s) [noise=true , address=104.36.82.43:4868 , piece_id=QSDRGPBV3KWFHHKWJWKLJ3N5APRHOI66TTWAWNOJDCRXQU2P7PAA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.865162168 (12h25m47.609219032s) [noise=true , address=104.36.82.60:4346 , piece_id=CJPTIG7EI264SA2JCH5326GTL33YW6ASGJPXO5ZRWRDD3L5BQTYA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.865201343 (12h25m47.609258205s) [seg_pos={0 0}, upload=12820]: Segment reached optimal threshold of 14 pieces. Aug 7 22:05:25.865362755 (12h25m47.609419601s) [noise=true , address=104.36.82.47:4534 , piece_id=JS7S2OPLDOU5C52ZACTR77R2OVC56PBN6BDNGAPRBNN3UHXQPX3Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VDrNGcCnaKEmG5NN4nex1JoZAgdCEfX5CdsKbxytfLfKzPbr): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865412734 (12h25m47.609469599s) [noise=true , address=104.36.82.20:4461 , piece_id=XGB6IA7NPLFF7XKYATSOG54O7U36IX6BVHVWGA53T7MNVBFM5R2A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nhwT2THUu6ptumqU9Bz1Nd34FHLewx9vWjGFC34j3Lz3RWfkw): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865468394 (12h25m47.609525241s) [noise=true , address=104.36.82.5:4862 , piece_id=W6AHEAV5E5RQR3YQ3LFOO33URBFGYOVTT23S5JKPP7MDNIRFHV5Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865629517 (12h25m47.609686385s) [noise=true , address=104.36.82.22:4530 , piece_id=U33JUCE4QFPT54RRVY57EF64QS7C4TUO7MWOOGZFAIHBZQUZFB3A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1akHUSoQFafzkGTJ385iYaowc87pJBDNfU5hodSCUDFa5ojN3v): protocol: write tcp 139.59.144.22:45386->104.36.82.22:4530: use of closed network connection; context canceled Aug 7 22:05:25.865657252 (12h25m47.609714121s) [seg_pos={0 0}, upload=12820]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:05:25.865814825 (12h25m47.609871688s) [upload=12820]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:05:25.865822925 (12h25m47.609879795s) [upload=12820]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:26.138690358 (12h25m47.882747231s) [upload=12820]: Done uploading segments... Aug 7 22:05:26.138713087 (12h25m47.882769948s) [upload=12820]: Upload finished. err: <nil> Duration: 3.678324125s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 21 out of 60 is failed (threshold=3) [XXXXXXXXXXXX......X................................X.XXXXXXX] duration: 1m0.000700496s FAIL (2025-08-07 22:19:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 173505984s) [upload=12891]: Done starting upload Aug 7 22:09:40.429576700 (12h30m2.173633561s) [upload=12891]: Uploading segments... Aug 7 22:09:40.429631265 (12h30m2.173688133s) [upload=12891]: Got next segment. Inline: true Aug 7 22:09:40.429637613 (12h30m2.173694464s) [upload=12891]: Waiting for error group managing segments... Aug 7 22:09:40.429689508 (12h30m2.173746362s) [upload=12891]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:09:40.429698290 (12h30m2.17375514s) [upload=12891]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:09:40.429702162 (12h30m2.173759014s) [upload=12891]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:09:40.614648207 (12h30m2.358705075s) [upload=12891]: Done uploading segments... Aug 7 22:09:40.614672259 (12h30m2.358729124s) [upload=12891]: Upload finished. err: <nil> Aug 7 22:10:28.314751006 (12h30m50.058807867s) [upload=12903]: Starting upload Aug 7 22:10:28.314813386 (12h30m50.058870247s) [upload=12903]: Done starting upload Aug 7 22:10:28.314903941 (12h30m50.058960802s) [upload=12903]: Uploading segments... Aug 7 22:10:28.314992884 (12h30m50.059049757s) [upload=12903]: Got next segment. Inline: true Aug 7 22:10:28.315015785 (12h30m50.059072647s) [upload=12903]: Waiting for error group managing segments... Aug 7 22:10:28.315072549 (12h30m50.059129415s) [upload=12903]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:10:28.315079820 (12h30m50.059136685s) [upload=12903]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:10:28.315099592 (12h30m50.059156457s) [upload=12903]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:11:28.039892681 (12h31m49.783949531s) [upload=12903]: Done uploading segments... Aug 7 22:11:28.039913432 (12h31m49.783970283s) [upload=12903]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:11:35.297516727 (12h31m57.041573588s) [upload=12923]: Starting upload Aug 7 22:11:35.297601596 (12h31m57.041658469s) [upload=12923]: Done starting upload Aug 7 22:11:35.297686234 (12h31m57.041743086s) [upload=12923]: Uploading segments... Aug 7 22:11:35.297779680 (12h31m57.041836543s) [upload=12923]: Got next segment. Inline: true Aug 7 22:11:35.297790270 (12h31m57.041847121s) [upload=12923]: Waiting for error group managing segments... Aug 7 22:11:35.297854321 (12h31m57.041911182s) [upload=12923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:11:35.297860387 (12h31m57.041917249s) [upload=12923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:11:35.297863885 (12h31m57.041920745s) [upload=12923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:11:44.558966532 (12h32m6.303023384s) [upload=12923]: Done uploading segments... Aug 7 22:11:44.558985310 (12h32m6.303042163s) [upload=12923]: Upload finished. err: <nil> Aug 7 22:12:36.559571434 (12h32m58.30362831s) [upload=12938]: Starting upload Aug 7 22:12:36.559699331 (12h32m58.3037562s) [upload=12938]: Done starting upload Aug 7 22:12:36.559803583 (12h32m58.30386044s) [upload=12938]: Uploading segments... Aug 7 22:12:36.560034082 (12h32m58.304090945s) [upload=12938]: Got next segment. Inline: true Aug 7 22:12:36.560041392 (12h32m58.304098252s) [upload=12938]: Waiting for error group managing segments... Aug 7 22:12:36.560151251 (12h32m58.304208113s) [upload=12938]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:12:36.560159993 (12h32m58.304216852s) [upload=12938]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:12:36.560164091 (12h32m58.304220952s) [upload=12938]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:13:36.275827133 (12h33m58.019883992s) [upload=12938]: Done uploading segments... Aug 7 22:13:36.275867976 (12h33m58.019924837s) [upload=12938]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:13:36.703916041 (12h33m58.447972892s) [upload=12954]: Starting upload Aug 7 22:13:36.704016678 (12h33m58.448073543s) [upload=12954]: Done starting upload Aug 7 22:13:36.704126028 (12h33m58.448182877s) [upload=12954]: Uploading segments... Aug 7 22:13:36.704202003 (12h33m58.448258875s) [upload=12954]: Got next segment. Inline: true Aug 7 22:13:36.704208724 (12h33m58.448265583s) [upload=12954]: Waiting for error group managing segments... Aug 7 22:13:36.704253077 (12h33m58.44830994s) [upload=12954]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:13:36.704259505 (12h33m58.448316353s) [upload=12954]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:13:36.704263329 (12h33m58.448320184s) [upload=12954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:14:36.426520849 (12h34m58.170577716s) [upload=12954]: Done uploading segments... Aug 7 22:14:36.426547324 (12h34m58.170604188s) [upload=12954]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:14:36.851362491 (12h34m58.595419352s) [upload=12969]: Starting upload Aug 7 22:14:36.851438058 (12h34m58.595494919s) [upload=12969]: Done starting upload Aug 7 22:14:36.851531336 (12h34m58.595588182s) [upload=12969]: Uploading segments... Aug 7 22:14:36.851620028 (12h34m58.595676898s) [upload=12969]: Got next segment. Inline: true Aug 7 22:14:36.851627191 (12h34m58.595684056s) [upload=12969]: Waiting for error group managing segments... Aug 7 22:14:36.851698724 (12h34m58.595755585s) [upload=12969]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:14:36.851706200 (12h34m58.595763081s) [upload=12969]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:14:36.851710849 (12h34m58.595767717s) [upload=12969]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:15:36.582802839 (12h35m58.326859693s) [upload=12969]: Done uploading segments... Aug 7 22:15:36.582825807 (12h35m58.326882658s) [upload=12969]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:37.013505363 (12h35m58.757562231s) [upload=12985]: Starting upload Aug 7 22:15:37.013605304 (12h35m58.757662168s) [upload=12985]: Done starting upload Aug 7 22:15:37.013715315 (12h35m58.757772164s) [upload=12985]: Uploading segments... Aug 7 22:15:37.013761212 (12h35m58.757818081s) [upload=12985]: Got next segment. Inline: true Aug 7 22:15:37.013776983 (12h35m58.757833843s) [upload=12985]: Waiting for error group managing segments... Aug 7 22:15:37.013820691 (12h35m58.757877552s) [upload=12985]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:15:37.013826603 (12h35m58.757883467s) [upload=12985]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:15:37.013830513 (12h35m58.757887374s) [upload=12985]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:16:36.733288222 (12h36m58.477345085s) [upload=12985]: Done uploading segments... Aug 7 22:16:36.733314893 (12h36m58.477371754s) [upload=12985]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:16:37.187949023 (12h36m58.932005874s) [upload=13003]: Starting upload Aug 7 22:16:37.188024595 (12h36m58.932081456s) [upload=13003]: Done starting upload Aug 7 22:16:37.188115827 (12h36m58.932172677s) [upload=13003]: Uploading segments... Aug 7 22:16:37.188173315 (12h36m58.932230182s) [upload=13003]: Got next segment. Inline: true Aug 7 22:16:37.188181498 (12h36m58.932238365s) [upload=13003]: Waiting for error group managing segments... Aug 7 22:16:37.188252566 (12h36m58.932309427s) [upload=13003]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:16:37.188260045 (12h36m58.932316913s) [upload=13003]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:16:37.188263607 (12h36m58.932320469s) [upload=13003]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:17:36.907306861 (12h37m58.651363739s) [upload=13003]: Done uploading segments... Aug 7 22:17:36.907340408 (12h37m58.651397275s) [upload=13003]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:46.749223731 (12h38m8.493280581s) [upload=13019]: Starting upload Aug 7 22:17:46.749288163 (12h38m8.493345017s) [upload=13019]: Done starting upload Aug 7 22:17:46.749381044 (12h38m8.493437894s) [upload=13019]: Uploading segments... Aug 7 22:17:46.749423663 (12h38m8.493480532s) [upload=13019]: Got next segment. Inline: true Aug 7 22:17:46.749428712 (12h38m8.493485573s) [upload=13019]: Waiting for error group managing segments... Aug 7 22:17:46.749471140 (12h38m8.49352799s) [upload=13019]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:17:46.749476259 (12h38m8.493533119s) [upload=13019]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:46.749479952 (12h38m8.493536802s) [upload=13019]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:18:46.471549080 (12h39m8.215605941s) [upload=13019]: Done uploading segments... Aug 7 22:18:46.471571547 (12h39m8.215628407s) [upload=13019]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:46.904761993 (12h39m8.648818857s) [upload=13033]: Starting upload Aug 7 22:18:46.904856076 (12h39m8.648912937s) [upload=13033]: Done starting upload Aug 7 22:18:46.904960146 (12h39m8.649017s) [upload=13033]: Uploading segments... Aug 7 22:18:46.905047245 (12h39m8.649104114s) [upload=13033]: Got next segment. Inline: true Aug 7 22:18:46.905056198 (12h39m8.649113053s) [upload=13033]: Waiting for error group managing segments... Aug 7 22:18:46.905116687 (12h39m8.649173538s) [upload=13033]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:18:46.905122460 (12h39m8.649179309s) [upload=13033]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:46.905126367 (12h39m8.649183281s) [upload=13033]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:19:46.626010888 (12h40m8.370067739s) [upload=13033]: Done uploading segments... Aug 7 22:19:46.626032620 (12h40m8.370089471s) [upload=13033]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.000728176s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................X...X..........................] duration: 940.318979ms FAIL (2025-08-07 22:19:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 12661]: Done uploading segments... Aug 7 21:56:36.224982877 (12h16m57.969039745s) [upload=12661]: Upload finished. err: <nil> Aug 7 21:57:38.819941245 (12h18m0.563998096s) [upload=12678]: Starting upload Aug 7 21:57:38.820005339 (12h18m0.564062199s) [upload=12678]: Done starting upload Aug 7 21:57:38.820093525 (12h18m0.564150382s) [upload=12678]: Uploading segments... Aug 7 21:57:38.820164980 (12h18m0.564221849s) [upload=12678]: Got next segment. Inline: true Aug 7 21:57:38.820170360 (12h18m0.564227223s) [upload=12678]: Waiting for error group managing segments... Aug 7 21:57:38.820217982 (12h18m0.564274832s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:38.820223803 (12h18m0.564280662s) [upload=12678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:38.820227203 (12h18m0.564284063s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:39.040341313 (12h18m0.784398161s) [upload=12678]: Done uploading segments... Aug 7 21:57:39.040360634 (12h18m0.784417481s) [upload=12678]: Upload finished. err: <nil> Aug 7 21:57:54.880649336 (12h18m16.624706187s) [upload=12684]: Starting upload Aug 7 21:57:54.880693218 (12h18m16.624750088s) [upload=12684]: Done starting upload Aug 7 21:57:54.880791132 (12h18m16.624847993s) [upload=12684]: Uploading segments... Aug 7 21:57:54.880832914 (12h18m16.624889781s) [upload=12684]: Got next segment. Inline: true Aug 7 21:57:54.880838436 (12h18m16.624895296s) [upload=12684]: Waiting for error group managing segments... Aug 7 21:57:54.880894572 (12h18m16.624951432s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:54.880902144 (12h18m16.624959006s) [upload=12684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:54.880905596 (12h18m16.624962457s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:55.045748011 (12h18m16.789804872s) [upload=12684]: Done uploading segments... Aug 7 21:57:55.045767142 (12h18m16.789823992s) [upload=12684]: Upload finished. err: <nil> Aug 7 21:58:56.209662862 (12h19m17.953719708s) [upload=12702]: Starting upload Aug 7 21:58:56.209717729 (12h19m17.953774589s) [upload=12702]: Done starting upload Aug 7 21:58:56.209791551 (12h19m17.953848398s) [upload=12702]: Uploading segments... Aug 7 21:58:56.209845084 (12h19m17.953901954s) [upload=12702]: Got next segment. Inline: true Aug 7 21:58:56.209850002 (12h19m17.953906863s) [upload=12702]: Waiting for error group managing segments... Aug 7 21:58:56.209900221 (12h19m17.953957082s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:58:56.209905576 (12h19m17.953962423s) [upload=12702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:58:56.209940267 (12h19m17.953997127s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:58:56.605688962 (12h19m18.349745811s) [upload=12702]: Done uploading segments... Aug 7 21:58:56.605708466 (12h19m18.349765315s) [upload=12702]: Upload finished. err: <nil> Aug 7 21:59:15.802505476 (12h19m37.546562336s) [upload=12707]: Starting upload Aug 7 21:59:15.802568926 (12h19m37.546625778s) [upload=12707]: Done starting upload Aug 7 21:59:15.802684030 (12h19m37.546740881s) [upload=12707]: Uploading segments... Aug 7 21:59:15.802727059 (12h19m37.546783925s) [upload=12707]: Got next segment. Inline: true Aug 7 21:59:15.802732418 (12h19m37.546789279s) [upload=12707]: Waiting for error group managing segments... Aug 7 21:59:15.802792878 (12h19m37.546849738s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:15.802798596 (12h19m37.546855456s) [upload=12707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:15.802802334 (12h19m37.546859194s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:15.945145817 (12h19m37.689202664s) [upload=12707]: Done uploading segments... Aug 7 21:59:15.945175582 (12h19m37.689232429s) [upload=12707]: Upload finished. err: <nil> Aug 7 21:59:31.540495488 (12h19m53.284552336s) [upload=12712]: Starting upload Aug 7 21:59:31.540551951 (12h19m53.284608811s) [upload=12712]: Done starting upload Aug 7 21:59:31.540628497 (12h19m53.284685347s) [upload=12712]: Uploading segments... Aug 7 21:59:31.540671158 (12h19m53.284728027s) [upload=12712]: Got next segment. Inline: true Aug 7 21:59:31.540677279 (12h19m53.284734139s) [upload=12712]: Waiting for error group managing segments... Aug 7 21:59:31.540730297 (12h19m53.284787147s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:31.540735969 (12h19m53.284792829s) [upload=12712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:31.540739244 (12h19m53.284796105s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:31.703548635 (12h19m53.447605493s) [upload=12712]: Done uploading segments... Aug 7 21:59:31.703590031 (12h19m53.447646882s) [upload=12712]: Upload finished. err: <nil> Aug 7 22:00:29.248229344 (12h20m50.992286195s) [upload=12728]: Starting upload Aug 7 22:00:29.248281332 (12h20m50.992338188s) [upload=12728]: Done starting upload Aug 7 22:00:29.248374886 (12h20m50.992431738s) [upload=12728]: Uploading segments... Aug 7 22:00:29.248452896 (12h20m50.992509768s) [upload=12728]: Got next segment. Inline: true Aug 7 22:00:29.248460057 (12h20m50.992516919s) [upload=12728]: Waiting for error group managing segments... Aug 7 22:00:29.248538087 (12h20m50.992594934s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:29.248545868 (12h20m50.992602715s) [upload=12728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:29.248549647 (12h20m50.992606513s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:29.574503821 (12h20m51.318560681s) [upload=12728]: Done uploading segments... Aug 7 22:00:29.574527802 (12h20m51.318584666s) [upload=12728]: Upload finished. err: <nil> Aug 7 22:01:13.393836672 (12h21m35.137893523s) [upload=12740]: Starting upload Aug 7 22:01:13.393888179 (12h21m35.137945042s) [upload=12740]: Done starting upload Aug 7 22:01:13.394016213 (12h21m35.138073077s) [upload=12740]: Uploading segments... Aug 7 22:01:13.394161981 (12h21m35.13821896s) [upload=12740]: Got next segment. Inline: true Aug 7 22:01:13.394170576 (12h21m35.138227445s) [upload=12740]: Waiting for error group managing segments... Aug 7 22:01:13.394246764 (12h21m35.13830363s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:13.394254537 (12h21m35.138311402s) [upload=12740]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:13.394274292 (12h21m35.138331152s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:13.707894690 (12h21m35.451951555s) [upload=12740]: Done uploading segments... Aug 7 22:01:13.707926477 (12h21m35.451983343s) [upload=12740]: Upload finished. err: <nil> Aug 7 22:01:29.783138431 (12h21m51.527195291s) [upload=12746]: Starting upload Aug 7 22:01:29.783212111 (12h21m51.527268976s) [upload=12746]: Done starting upload Aug 7 22:01:29.783446600 (12h21m51.52750345s) [upload=12746]: Uploading segments... Aug 7 22:01:29.783527384 (12h21m51.527584254s) [upload=12746]: Got next segment. Inline: true Aug 7 22:01:29.783533979 (12h21m51.52759083s) [upload=12746]: Waiting for error group managing segments... Aug 7 22:01:29.783596800 (12h21m51.527653652s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:29.783605977 (12h21m51.527662827s) [upload=12746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:29.783609731 (12h21m51.52766658s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:29.966350494 (12h21m51.710407342s) [upload=12746]: Done uploading segments... Aug 7 22:01:29.966369237 (12h21m51.710426083s) [upload=12746]: Upload finished. err: <nil> Aug 7 22:01:45.372266942 (12h22m7.116323788s) [upload=12750]: Starting upload Aug 7 22:01:45.372314023 (12h22m7.116370883s) [upload=12750]: Done starting upload Aug 7 22:01:45.372374655 (12h22m7.116431503s) [upload=12750]: Uploading segments... Aug 7 22:01:45.372417267 (12h22m7.116474136s) [upload=12750]: Got next segment. Inline: true Aug 7 22:01:45.372422121 (12h22m7.116478981s) [upload=12750]: Waiting for error group managing segments... Aug 7 22:01:45.372461831 (12h22m7.116518692s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:45.372466235 (12h22m7.116523094s) [upload=12750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:45.372468880 (12h22m7.11652574s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:45.597843459 (12h22m7.341900345s) [upload=12750]: Done uploading segments... Aug 7 22:01:45.597877236 (12h22m7.341934095s) [upload=12750]: Upload finished. err: <nil> Aug 7 22:02:22.792150325 (12h22m44.53620717s) [upload=12764]: Starting upload Aug 7 22:02:22.792208195 (12h22m44.536265055s) [upload=12764]: Done starting upload Aug 7 22:02:22.792266980 (12h22m44.536323827s) [upload=12764]: Uploading segments... Aug 7 22:02:22.792307216 (12h22m44.536364085s) [upload=12764]: Got next segment. Inline: true Aug 7 22:02:22.792311926 (12h22m44.536368786s) [upload=12764]: Waiting for error group managing segments... Aug 7 22:02:22.792351265 (12h22m44.536408125s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.792356546 (12h22m44.536413408s) [upload=12764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.792359208 (12h22m44.536416068s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:23.139246613 (12h22m44.883303464s) [upload=12764]: Done uploading segments... Aug 7 22:02:23.139265320 (12h22m44.883322173s) [upload=12764]: Upload finished. err: <nil> Duration: 940.347818ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 24 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXX....................................XXXXXXXXXX] duration: 30.001568807s FAIL (2025-08-07 22:19:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 12h31m49.198871679s) [upload=12920]: Uploading segments... Aug 7 22:11:27.454893840 (12h31m49.198950813s) [upload=12920]: Got next segment. Inline: true Aug 7 22:11:27.454915725 (12h31m49.198972587s) [upload=12920]: Waiting for error group managing segments... Aug 7 22:11:27.454970562 (12h31m49.199027411s) [upload=12920]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:11:27.454977263 (12h31m49.199034131s) [upload=12920]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:11:27.455074233 (12h31m49.199131094s) [upload=12920]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:11:57.179550579 (12h32m18.92360743s) [upload=12920]: Done uploading segments... Aug 7 22:11:57.179569786 (12h32m18.923626636s) [upload=12920]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:12:19.183043096 (12h32m40.927099946s) [upload=12933]: Starting upload Aug 7 22:12:19.183127401 (12h32m40.927184261s) [upload=12933]: Done starting upload Aug 7 22:12:19.183328044 (12h32m40.92738489s) [upload=12933]: Uploading segments... Aug 7 22:12:19.183400189 (12h32m40.92745706s) [upload=12933]: Got next segment. Inline: true Aug 7 22:12:19.183407329 (12h32m40.927464177s) [upload=12933]: Waiting for error group managing segments... Aug 7 22:12:19.183479311 (12h32m40.927536158s) [upload=12933]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:12:19.183484643 (12h32m40.927541506s) [upload=12933]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:12:19.183487379 (12h32m40.927544225s) [upload=12933]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:12:48.909423390 (12h33m10.653480257s) [upload=12933]: Done uploading segments... Aug 7 22:12:48.909444894 (12h33m10.653501768s) [upload=12933]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:13:10.831892676 (12h33m32.575949525s) [upload=12948]: Starting upload Aug 7 22:13:10.831993207 (12h33m32.576050068s) [upload=12948]: Done starting upload Aug 7 22:13:10.832071605 (12h33m32.576128457s) [upload=12948]: Uploading segments... Aug 7 22:13:10.832146431 (12h33m32.576203291s) [upload=12948]: Got next segment. Inline: true Aug 7 22:13:10.832152814 (12h33m32.576209674s) [upload=12948]: Waiting for error group managing segments... Aug 7 22:13:10.832208966 (12h33m32.576265827s) [upload=12948]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:13:10.832214384 (12h33m32.576271244s) [upload=12948]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:13:10.832217904 (12h33m32.576274764s) [upload=12948]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:13:40.559928781 (12h34m2.303985629s) [upload=12948]: Done uploading segments... Aug 7 22:13:40.559950271 (12h34m2.304007117s) [upload=12948]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:13:48.685772427 (12h34m10.429829287s) [upload=12958]: Starting upload Aug 7 22:13:48.685858893 (12h34m10.429915753s) [upload=12958]: Done starting upload Aug 7 22:13:48.685982193 (12h34m10.430039042s) [upload=12958]: Uploading segments... Aug 7 22:13:48.686047966 (12h34m10.430104835s) [upload=12958]: Got next segment. Inline: true Aug 7 22:13:48.686053563 (12h34m10.430110423s) [upload=12958]: Waiting for error group managing segments... Aug 7 22:13:48.686096300 (12h34m10.43015316s) [upload=12958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:13:48.686101341 (12h34m10.430158201s) [upload=12958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:13:48.686104243 (12h34m10.430161104s) [upload=12958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:14:18.399606594 (12h34m40.143663447s) [upload=12958]: Done uploading segments... Aug 7 22:14:18.399627822 (12h34m40.143684675s) [upload=12958]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:14:59.902619970 (12h35m21.646676816s) [upload=12977]: Starting upload Aug 7 22:14:59.902685895 (12h35m21.646742756s) [upload=12977]: Done starting upload Aug 7 22:14:59.902751694 (12h35m21.646808545s) [upload=12977]: Uploading segments... Aug 7 22:14:59.902795346 (12h35m21.646852214s) [upload=12977]: Got next segment. Inline: true Aug 7 22:14:59.902801495 (12h35m21.646858346s) [upload=12977]: Waiting for error group managing segments... Aug 7 22:14:59.902858709 (12h35m21.64691556s) [upload=12977]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:14:59.902863962 (12h35m21.646920811s) [upload=12977]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:14:59.902867458 (12h35m21.646924309s) [upload=12977]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:15:29.636302762 (12h35m51.380359612s) [upload=12977]: Done uploading segments... Aug 7 22:15:29.636325398 (12h35m51.380382249s) [upload=12977]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:45.428850713 (12h36m7.172907563s) [upload=12988]: Starting upload Aug 7 22:15:45.428978829 (12h36m7.173035696s) [upload=12988]: Done starting upload Aug 7 22:15:45.429103446 (12h36m7.173160298s) [upload=12988]: Uploading segments... Aug 7 22:15:45.429150754 (12h36m7.173207623s) [upload=12988]: Got next segment. Inline: true Aug 7 22:15:45.429156339 (12h36m7.173213199s) [upload=12988]: Waiting for error group managing segments... Aug 7 22:15:45.429230361 (12h36m7.173287221s) [upload=12988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:15:45.429247463 (12h36m7.173304325s) [upload=12988]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:15:45.429251133 (12h36m7.173307993s) [upload=12988]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:16:15.155440875 (12h36m36.899497725s) [upload=12988]: Done uploading segments... Aug 7 22:16:15.155471506 (12h36m36.899528367s) [upload=12988]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:16:15.598565945 (12h36m37.342622791s) [upload=12999]: Starting upload Aug 7 22:16:15.598645050 (12h36m37.342701913s) [upload=12999]: Done starting upload Aug 7 22:16:15.598718650 (12h36m37.342775497s) [upload=12999]: Uploading segments... Aug 7 22:16:15.598774264 (12h36m37.34283113s) [upload=12999]: Got next segment. Inline: true Aug 7 22:16:15.598779124 (12h36m37.342835984s) [upload=12999]: Waiting for error group managing segments... Aug 7 22:16:15.598830461 (12h36m37.342887324s) [upload=12999]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:16:15.598854102 (12h36m37.342910963s) [upload=12999]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:16:15.598857833 (12h36m37.342914694s) [upload=12999]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:16:45.328584439 (12h37m7.072641289s) [upload=12999]: Done uploading segments... Aug 7 22:16:45.328609005 (12h37m7.072665856s) [upload=12999]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:27.480756310 (12h37m49.22481316s) [upload=13015]: Starting upload Aug 7 22:17:27.480859326 (12h37m49.22491619s) [upload=13015]: Done starting upload Aug 7 22:17:27.480944450 (12h37m49.225001301s) [upload=13015]: Uploading segments... Aug 7 22:17:27.481000891 (12h37m49.225057757s) [upload=13015]: Got next segment. Inline: true Aug 7 22:17:27.481006468 (12h37m49.225063328s) [upload=13015]: Waiting for error group managing segments... Aug 7 22:17:27.481069727 (12h37m49.225126586s) [upload=13015]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:17:27.481076622 (12h37m49.225133484s) [upload=13015]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:27.481081288 (12h37m49.225138155s) [upload=13015]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:17:57.211870572 (12h38m18.955927423s) [upload=13015]: Done uploading segments... Aug 7 22:17:57.211892595 (12h38m18.955949444s) [upload=13015]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:14.331617780 (12h38m36.07567463s) [upload=13026]: Starting upload Aug 7 22:18:14.331690520 (12h38m36.07574737s) [upload=13026]: Done starting upload Aug 7 22:18:14.331789874 (12h38m36.075846724s) [upload=13026]: Uploading segments... Aug 7 22:18:14.331829497 (12h38m36.075886363s) [upload=13026]: Got next segment. Inline: true Aug 7 22:18:14.331834588 (12h38m36.075891448s) [upload=13026]: Waiting for error group managing segments... Aug 7 22:18:14.331890028 (12h38m36.075946878s) [upload=13026]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:18:14.331895594 (12h38m36.075952444s) [upload=13026]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:14.331899378 (12h38m36.075956238s) [upload=13026]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:18:44.050649404 (12h39m5.794706254s) [upload=13026]: Done uploading segments... Aug 7 22:18:44.050671176 (12h39m5.794728026s) [upload=13026]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:08.682705115 (12h39m30.42676198s) [upload=13039]: Starting upload Aug 7 22:19:08.682797967 (12h39m30.426854836s) [upload=13039]: Done starting upload Aug 7 22:19:08.682924116 (12h39m30.426980968s) [upload=13039]: Uploading segments... Aug 7 22:19:08.682986486 (12h39m30.427043358s) [upload=13039]: Got next segment. Inline: true Aug 7 22:19:08.682993964 (12h39m30.427050856s) [upload=13039]: Waiting for error group managing segments... Aug 7 22:19:08.683054026 (12h39m30.427110894s) [upload=13039]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:19:08.683060373 (12h39m30.427117244s) [upload=13039]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:08.683064440 (12h39m30.42712131s) [upload=13039]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:19:38.406368834 (12h40m0.150425697s) [upload=13039]: Done uploading segments... Aug 7 22:19:38.406426692 (12h40m0.150483554s) [upload=13039]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.001611536s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 22 out of 60 is failed (threshold=3) [XXXXXXXX..X.XX..................................X.XXXXXXXXXX] duration: 30.000856135s FAIL (2025-08-07 22:19:37 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000878483s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 26 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXX......X.X...........................XXXXXXXX] duration: 1m0.000567544s FAIL (2025-08-07 22:19:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000580724s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 17 out of 60 is failed (threshold=3) [XXXXXXXX.X..........................................XXXXXXXX] duration: 30.000873816s FAIL (2025-08-07 22:19:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000908012s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [.......XX.X................................................X] duration: 2m0.000387416s remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:19:20 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 2m0.000404035s edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [.X.X..................................................X.....] duration: 305.744812ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:19:17 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 305.765413ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXX.....................................XXXXXXXX] duration: 30.00087825s FAIL (2025-08-07 22:19:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000909836s edge gateway us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 104.151599ms remote: "tcp:185.244.226.2:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-07 22:19:15 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 104.180556ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [.....................XXXX..............................XXXXX] duration: 1m0.000784162s FAIL (2025-08-07 22:19:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== QTTJN77TZIQ, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.396205503 (12h32m18.140262364s) [noise=true , address=45.91.116.9:4272 , piece_id=3QGHJZTVECLQTJGC46GZ5276LVTHR36LBTK5Q5FSRGD7QXOUJJ6A, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.397743950 (12h32m18.141800804s) [noise=true , address=64.49.13.139:4471 , piece_id=QTORROJDLNZQTDC2VFU3Q5DW47TA7VM7FZAVKTACLLCPMHGOC52A, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.398091992 (12h32m18.142148846s) [noise=true , address=104.36.82.43:4837 , piece_id=WALP4A6IL6VR6577XHAIATXZGBECNO6BL5Y5Z4I4DMPCNIK7X5QQ, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.979185930 (12h32m18.723242776s) [upload=12925]: Next returned nil segment Aug 7 22:11:56.979206213 (12h32m18.723263074s) [upload=12925]: Waiting for error group managing segments... Aug 7 22:11:57.134173791 (12h32m18.878230638s) [noise=true , address=104.36.82.13:4188 , piece_id=C24Q324TD3OIGJSBOKVUN2VH5ORNKINWDHBTA3O56VSRPJN52SGA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.151552906 (12h32m18.895609777s) [noise=true , address=104.36.82.6:4895 , piece_id=HUPRXOSI2CS2UDOF47BME2BYCCLXANUCJAYVZV4JNRMPQQUDZQFA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.151985548 (12h32m18.896042414s) [noise=true , address=104.36.82.60:4343 , piece_id=A5ZQY7FYEN7BVIGGB6T43UTFNOC6NGPBM3XJ23QSLVRFFVSVYJCQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.177078648 (12h32m18.921135515s) [noise=true , address=104.36.82.34:4566 , piece_id=GR2IDGJL2TB5VYWZOIW2C652I5B5XUUXXI4SIELMP3V4GMO2ZH4Q, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.180968885 (12h32m18.925025749s) [noise=true , address=104.36.82.45:4940 , piece_id=E2BSUMWRGNFOCNBBT32GAMW3JN4PFKNKSYXK5AEUGQQKRK4I6FKA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.182573730 (12h32m18.926630601s) [noise=true , address=104.36.82.43:4837 , piece_id=WALP4A6IL6VR6577XHAIATXZGBECNO6BL5Y5Z4I4DMPCNIK7X5QQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.185217873 (12h32m18.929274727s) [noise=true , address=104.36.82.60:4346 , piece_id=OCI6XALWGEOAF64WRNURL5VVPFWYBT24GXK4GNQMUX2MJYZACROQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.191081154 (12h32m18.935138022s) [noise=true , address=104.36.82.23:4448 , piece_id=RD4YTZPX2A4SRE2XWYS45DWZRG6CUS5VLCYBMJPBJ24LGJTGITFQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.205956300 (12h32m18.950013175s) [noise=true , address=104.36.82.36:4645 , piece_id=P63BF2T4FLUK7DMM4GQY277QTDGZXT3CD4Z5GFDMUQTTJN77TZIQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.214668342 (12h32m18.958725212s) [noise=true , address=104.36.82.39:4733 , piece_id=VZPW75CLRYOWCVQGWBWAOGWKFCXYLYOM6OIDMXRBR5IRGZ7DGLLQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.245950582 (12h32m18.990007453s) [noise=true , address=104.36.82.16:4310 , piece_id=UAVZCLFOGCC6T4HX676IX6OWFNRZRTXL3PITG5IZFR2JATIZS2DQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.280359350 (12h32m19.024416213s) [noise=true , address=45.91.116.9:4272 , piece_id=3QGHJZTVECLQTJGC46GZ5276LVTHR36LBTK5Q5FSRGD7QXOUJJ6A, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.338710901 (12h32m19.082767752s) [noise=true , address=45.91.116.13:4166 , piece_id=HSFXWLME6GIFAWHD73GW4H6CBPBJZ4MPTB7JD7XY2YBTZESICNRA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.617684582 (12h32m19.361741428s) [noise=true , address=104.36.82.62:4439 , piece_id=2P7NHL5WLDREVBGDS4IMRS6D4DZBQHRYFM7USCDK2KU74SKDLW4A, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.617723267 (12h32m19.361780128s) [seg_pos={0 0}, upload=12925]: Segment reached optimal threshold of 14 pieces. Aug 7 22:11:57.617992734 (12h32m19.36204958s) [noise=true , address=104.36.82.47:4520 , piece_id=DLEDG3JMSGGGFQFGNJWV4XIDPXITG54XCYTRXXJIRR4DOXWIDAVA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 22:11:57.618089635 (12h32m19.362146495s) [noise=true , address=64.49.13.137:4498 , piece_id=7SLUBM4U2CKK25FCFPHVKBA4D4TKCRZXKASC7LR2ASBM7C7MDQKQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: expected piece hash; context canceled; EOF Aug 7 22:11:57.618135706 (12h32m19.362192556s) [noise=true , address=104.36.82.5:4855 , piece_id=OEDGYJGQIIJHLEBR5OPPQCO75KXIGPFZBDCCTBYSUV7XQADJRSAA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh): protocol: write tcp 139.59.144.22:57090->104.36.82.5:4855: use of closed network connection; write tcp 139.59.144.22:57090->104.36.82.5:4855: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57090->104.36.82.5:4855: use of closed network connection Aug 7 22:11:57.618165938 (12h32m19.362222785s) [noise=true , address=64.49.13.139:4471 , piece_id=QTORROJDLNZQTDC2VFU3Q5DW47TA7VM7FZAVKTACLLCPMHGOC52A, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129YkXH84aYSubinAC57swW24qCCEtvJTSMDGoxP9WdgpGcZxLr): protocol: expected piece hash; context canceled; EOF Aug 7 22:11:57.618181168 (12h32m19.362238028s) [seg_pos={0 0}, upload=12925]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:11:57.618262981 (12h32m19.362319833s) [upload=12925]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:11:57.618268565 (12h32m19.362325425s) [upload=12925]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:15:11.462898274 (12h35m33.206955137s) [upload=12925]: Done uploading segments... Aug 7 22:15:11.462927061 (12h35m33.206983911s) [upload=12925]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:11.893865947 (12h35m33.637922798s) [upload=12978]: Starting upload Aug 7 22:15:11.894006161 (12h35m33.638063025s) [upload=12978]: Done starting upload Aug 7 22:15:11.894128456 (12h35m33.63818531s) [upload=12978]: Uploading segments... Aug 7 22:15:11.894439225 (12h35m33.638496094s) [upload=12978]: Got next segment. Inline: false Aug 7 22:15:11.894516777 (12h35m33.638573639s) [upload=12978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:15:11.894545833 (12h35m33.638602694s) [upload=12978]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:16:11.626077800 (12h36m33.370134666s) [upload=12978]: Next segment err: context deadline exceeded Aug 7 22:16:11.626165441 (12h36m33.370222301s) [upload=12978]: Done uploading segments... Aug 7 22:16:11.626183066 (12h36m33.370239913s) [upload=12978]: Upload finished. err: context deadline exceeded Aug 7 22:16:12.058909015 (12h36m33.802965875s) [upload=12997]: Starting upload Aug 7 22:16:12.058982372 (12h36m33.803039232s) [upload=12997]: Done starting upload Aug 7 22:16:12.059120653 (12h36m33.803177509s) [upload=12997]: Uploading segments... Aug 7 22:16:12.059593126 (12h36m33.803649985s) [upload=12997]: Got next segment. Inline: false Aug 7 22:16:12.059677172 (12h36m33.803734023s) [upload=12997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:16:12.059684424 (12h36m33.803741287s) [upload=12997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:17:11.787443939 (12h37m33.531500809s) [upload=12997]: Next segment err: context deadline exceeded Aug 7 22:17:11.787498695 (12h37m33.531555556s) [upload=12997]: Done uploading segments... Aug 7 22:17:11.787527052 (12h37m33.531583904s) [upload=12997]: Upload finished. err: context deadline exceeded Aug 7 22:17:12.226578214 (12h37m33.970635065s) [upload=13013]: Starting upload Aug 7 22:17:12.226666960 (12h37m33.970723825s) [upload=13013]: Done starting upload Aug 7 22:17:12.226793229 (12h37m33.970850079s) [upload=13013]: Uploading segments... Aug 7 22:17:12.227044345 (12h37m33.971101196s) [upload=13013]: Got next segment. Inline: false Aug 7 22:17:12.227107713 (12h37m33.971164563s) [upload=13013]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:17:12.227113591 (12h37m33.971170444s) [upload=13013]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:18:11.956251664 (12h38m33.700308534s) [upload=13013]: Next segment err: context deadline exceeded Aug 7 22:18:11.956360027 (12h38m33.700416877s) [upload=13013]: Done uploading segments... Aug 7 22:18:11.956379323 (12h38m33.700436185s) [upload=13013]: Upload finished. err: context deadline exceeded Aug 7 22:18:12.371524537 (12h38m34.115581388s) [upload=13024]: Starting upload Aug 7 22:18:12.371599713 (12h38m34.115656573s) [upload=13024]: Done starting upload Aug 7 22:18:12.371686548 (12h38m34.115743409s) [upload=13024]: Uploading segments... Aug 7 22:18:12.371885224 (12h38m34.115942074s) [upload=13024]: Got next segment. Inline: false Aug 7 22:18:12.371926111 (12h38m34.115982962s) [upload=13024]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:18:12.371931665 (12h38m34.115988525s) [upload=13024]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:19:12.105017725 (12h39m33.849074594s) [upload=13024]: Next segment err: context deadline exceeded Aug 7 22:19:12.105111880 (12h39m33.849168731s) [upload=13024]: Done uploading segments... Aug 7 22:19:12.105162034 (12h39m33.849218885s) [upload=13024]: Upload finished. err: context deadline exceeded Duration: 1m0.000803844s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................X...X.........................] duration: 1.039797169s FAIL (2025-08-07 22:19:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 12661]: Done uploading segments... Aug 7 21:56:36.224982877 (12h16m57.969039745s) [upload=12661]: Upload finished. err: <nil> Aug 7 21:57:38.819941245 (12h18m0.563998096s) [upload=12678]: Starting upload Aug 7 21:57:38.820005339 (12h18m0.564062199s) [upload=12678]: Done starting upload Aug 7 21:57:38.820093525 (12h18m0.564150382s) [upload=12678]: Uploading segments... Aug 7 21:57:38.820164980 (12h18m0.564221849s) [upload=12678]: Got next segment. Inline: true Aug 7 21:57:38.820170360 (12h18m0.564227223s) [upload=12678]: Waiting for error group managing segments... Aug 7 21:57:38.820217982 (12h18m0.564274832s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:38.820223803 (12h18m0.564280662s) [upload=12678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:38.820227203 (12h18m0.564284063s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:39.040341313 (12h18m0.784398161s) [upload=12678]: Done uploading segments... Aug 7 21:57:39.040360634 (12h18m0.784417481s) [upload=12678]: Upload finished. err: <nil> Aug 7 21:57:54.880649336 (12h18m16.624706187s) [upload=12684]: Starting upload Aug 7 21:57:54.880693218 (12h18m16.624750088s) [upload=12684]: Done starting upload Aug 7 21:57:54.880791132 (12h18m16.624847993s) [upload=12684]: Uploading segments... Aug 7 21:57:54.880832914 (12h18m16.624889781s) [upload=12684]: Got next segment. Inline: true Aug 7 21:57:54.880838436 (12h18m16.624895296s) [upload=12684]: Waiting for error group managing segments... Aug 7 21:57:54.880894572 (12h18m16.624951432s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:54.880902144 (12h18m16.624959006s) [upload=12684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:54.880905596 (12h18m16.624962457s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:55.045748011 (12h18m16.789804872s) [upload=12684]: Done uploading segments... Aug 7 21:57:55.045767142 (12h18m16.789823992s) [upload=12684]: Upload finished. err: <nil> Aug 7 21:58:56.209662862 (12h19m17.953719708s) [upload=12702]: Starting upload Aug 7 21:58:56.209717729 (12h19m17.953774589s) [upload=12702]: Done starting upload Aug 7 21:58:56.209791551 (12h19m17.953848398s) [upload=12702]: Uploading segments... Aug 7 21:58:56.209845084 (12h19m17.953901954s) [upload=12702]: Got next segment. Inline: true Aug 7 21:58:56.209850002 (12h19m17.953906863s) [upload=12702]: Waiting for error group managing segments... Aug 7 21:58:56.209900221 (12h19m17.953957082s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:58:56.209905576 (12h19m17.953962423s) [upload=12702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:58:56.209940267 (12h19m17.953997127s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:58:56.605688962 (12h19m18.349745811s) [upload=12702]: Done uploading segments... Aug 7 21:58:56.605708466 (12h19m18.349765315s) [upload=12702]: Upload finished. err: <nil> Aug 7 21:59:15.802505476 (12h19m37.546562336s) [upload=12707]: Starting upload Aug 7 21:59:15.802568926 (12h19m37.546625778s) [upload=12707]: Done starting upload Aug 7 21:59:15.802684030 (12h19m37.546740881s) [upload=12707]: Uploading segments... Aug 7 21:59:15.802727059 (12h19m37.546783925s) [upload=12707]: Got next segment. Inline: true Aug 7 21:59:15.802732418 (12h19m37.546789279s) [upload=12707]: Waiting for error group managing segments... Aug 7 21:59:15.802792878 (12h19m37.546849738s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:15.802798596 (12h19m37.546855456s) [upload=12707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:15.802802334 (12h19m37.546859194s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:15.945145817 (12h19m37.689202664s) [upload=12707]: Done uploading segments... Aug 7 21:59:15.945175582 (12h19m37.689232429s) [upload=12707]: Upload finished. err: <nil> Aug 7 21:59:31.540495488 (12h19m53.284552336s) [upload=12712]: Starting upload Aug 7 21:59:31.540551951 (12h19m53.284608811s) [upload=12712]: Done starting upload Aug 7 21:59:31.540628497 (12h19m53.284685347s) [upload=12712]: Uploading segments... Aug 7 21:59:31.540671158 (12h19m53.284728027s) [upload=12712]: Got next segment. Inline: true Aug 7 21:59:31.540677279 (12h19m53.284734139s) [upload=12712]: Waiting for error group managing segments... Aug 7 21:59:31.540730297 (12h19m53.284787147s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:31.540735969 (12h19m53.284792829s) [upload=12712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:31.540739244 (12h19m53.284796105s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:31.703548635 (12h19m53.447605493s) [upload=12712]: Done uploading segments... Aug 7 21:59:31.703590031 (12h19m53.447646882s) [upload=12712]: Upload finished. err: <nil> Aug 7 22:00:29.248229344 (12h20m50.992286195s) [upload=12728]: Starting upload Aug 7 22:00:29.248281332 (12h20m50.992338188s) [upload=12728]: Done starting upload Aug 7 22:00:29.248374886 (12h20m50.992431738s) [upload=12728]: Uploading segments... Aug 7 22:00:29.248452896 (12h20m50.992509768s) [upload=12728]: Got next segment. Inline: true Aug 7 22:00:29.248460057 (12h20m50.992516919s) [upload=12728]: Waiting for error group managing segments... Aug 7 22:00:29.248538087 (12h20m50.992594934s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:29.248545868 (12h20m50.992602715s) [upload=12728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:29.248549647 (12h20m50.992606513s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:29.574503821 (12h20m51.318560681s) [upload=12728]: Done uploading segments... Aug 7 22:00:29.574527802 (12h20m51.318584666s) [upload=12728]: Upload finished. err: <nil> Aug 7 22:01:13.393836672 (12h21m35.137893523s) [upload=12740]: Starting upload Aug 7 22:01:13.393888179 (12h21m35.137945042s) [upload=12740]: Done starting upload Aug 7 22:01:13.394016213 (12h21m35.138073077s) [upload=12740]: Uploading segments... Aug 7 22:01:13.394161981 (12h21m35.13821896s) [upload=12740]: Got next segment. Inline: true Aug 7 22:01:13.394170576 (12h21m35.138227445s) [upload=12740]: Waiting for error group managing segments... Aug 7 22:01:13.394246764 (12h21m35.13830363s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:13.394254537 (12h21m35.138311402s) [upload=12740]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:13.394274292 (12h21m35.138331152s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:13.707894690 (12h21m35.451951555s) [upload=12740]: Done uploading segments... Aug 7 22:01:13.707926477 (12h21m35.451983343s) [upload=12740]: Upload finished. err: <nil> Aug 7 22:01:29.783138431 (12h21m51.527195291s) [upload=12746]: Starting upload Aug 7 22:01:29.783212111 (12h21m51.527268976s) [upload=12746]: Done starting upload Aug 7 22:01:29.783446600 (12h21m51.52750345s) [upload=12746]: Uploading segments... Aug 7 22:01:29.783527384 (12h21m51.527584254s) [upload=12746]: Got next segment. Inline: true Aug 7 22:01:29.783533979 (12h21m51.52759083s) [upload=12746]: Waiting for error group managing segments... Aug 7 22:01:29.783596800 (12h21m51.527653652s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:29.783605977 (12h21m51.527662827s) [upload=12746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:29.783609731 (12h21m51.52766658s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:29.966350494 (12h21m51.710407342s) [upload=12746]: Done uploading segments... Aug 7 22:01:29.966369237 (12h21m51.710426083s) [upload=12746]: Upload finished. err: <nil> Aug 7 22:01:45.372266942 (12h22m7.116323788s) [upload=12750]: Starting upload Aug 7 22:01:45.372314023 (12h22m7.116370883s) [upload=12750]: Done starting upload Aug 7 22:01:45.372374655 (12h22m7.116431503s) [upload=12750]: Uploading segments... Aug 7 22:01:45.372417267 (12h22m7.116474136s) [upload=12750]: Got next segment. Inline: true Aug 7 22:01:45.372422121 (12h22m7.116478981s) [upload=12750]: Waiting for error group managing segments... Aug 7 22:01:45.372461831 (12h22m7.116518692s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:45.372466235 (12h22m7.116523094s) [upload=12750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:45.372468880 (12h22m7.11652574s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:45.597843459 (12h22m7.341900345s) [upload=12750]: Done uploading segments... Aug 7 22:01:45.597877236 (12h22m7.341934095s) [upload=12750]: Upload finished. err: <nil> Aug 7 22:02:22.792150325 (12h22m44.53620717s) [upload=12764]: Starting upload Aug 7 22:02:22.792208195 (12h22m44.536265055s) [upload=12764]: Done starting upload Aug 7 22:02:22.792266980 (12h22m44.536323827s) [upload=12764]: Uploading segments... Aug 7 22:02:22.792307216 (12h22m44.536364085s) [upload=12764]: Got next segment. Inline: true Aug 7 22:02:22.792311926 (12h22m44.536368786s) [upload=12764]: Waiting for error group managing segments... Aug 7 22:02:22.792351265 (12h22m44.536408125s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.792356546 (12h22m44.536413408s) [upload=12764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.792359208 (12h22m44.536416068s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:23.139246613 (12h22m44.883303464s) [upload=12764]: Done uploading segments... Aug 7 22:02:23.139265320 (12h22m44.883322173s) [upload=12764]: Upload finished. err: <nil> Duration: 1.039824462s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [X..........X.....X.........X..X...........X.................] duration: 3.525373056s FAIL (2025-08-07 22:18:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 5.727759364s) [upload=12820]: Starting upload Aug 7 22:05:23.983754833 (12h25m45.727811679s) [upload=12820]: Done starting upload Aug 7 22:05:23.983988533 (12h25m45.728045379s) [upload=12820]: Uploading segments... Aug 7 22:05:23.984022348 (12h25m45.728079214s) [upload=12820]: Got next segment. Inline: false Aug 7 22:05:23.984055700 (12h25m45.728112548s) [upload=12820]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:23.984060538 (12h25m45.728117398s) [upload=12820]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:05:24.185392691 (12h25m45.929449561s) [seg_pos={0 0}, upload=12820]: Begin upload segment... Aug 7 22:05:24.185557956 (12h25m45.929614817s) [seg_pos={0 0}, upload=12820]: Done begin upload segment. Aug 7 22:05:24.185603301 (12h25m45.929660177s) [noise=true , address=64.49.13.169:4705 , piece_id=6C45BITH3QWBJ22PLPILYA6LOUQ2JF2D2LOAL2BVJQQWXSXQ37IQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186037040 (12h25m45.930093906s) [noise=true , address=104.36.82.12:4174 , piece_id=DCAYU4CKMHLHOHAN7MKLV5KJTRCCC2IOWSPFPWEMAU3FSIVIXRQA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186570414 (12h25m45.930627287s) [noise=true , address=104.36.82.16:4314 , piece_id=KX4ZY5LSNB6YBYAS4NKW2QQXXQZV42VGR2TRRKZIL6ZIO3M7BAQQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186824559 (12h25m45.930881425s) [noise=true , address=104.36.82.46:4496 , piece_id=XNRPMJSQ3BNIILAQDMRYLFM623WHRZ63ND4RDTHJYLSWWOPNJGHA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187208122 (12h25m45.931264991s) [noise=true , address=104.36.82.43:4868 , piece_id=QSDRGPBV3KWFHHKWJWKLJ3N5APRHOI66TTWAWNOJDCRXQU2P7PAA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187658761 (12h25m45.931715627s) [noise=true , address=104.36.82.60:4346 , piece_id=CJPTIG7EI264SA2JCH5326GTL33YW6ASGJPXO5ZRWRDD3L5BQTYA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187940002 (12h25m45.931996872s) [noise=true , address=64.49.13.140:4515 , piece_id=WQSSOLYXPIBGHNVEG5IWDPLM45QDWPATVU73OJJTRU7NJRQCHDKA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188116203 (12h25m45.932173069s) [noise=true , address=45.91.116.6:4982 , piece_id=LAWUMX2HZNNHRQ3FUODV65COBDIMOZN2HBCSPD42P5DEC4N4VABQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188531115 (12h25m45.932587976s) [noise=true , address=104.36.82.58:4292 , piece_id=RJJS3MVCC3B3RTAXEM3XLHRSWZEUKIJUSILSKJKT5LJCWNEHX5GA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188637273 (12h25m45.932694145s) [noise=true , address=104.36.82.5:4862 , piece_id=W6AHEAV5E5RQR3YQ3LFOO33URBFGYOVTT23S5JKPP7MDNIRFHV5Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188961006 (12h25m45.933017872s) [noise=true , address=104.36.82.22:4530 , piece_id=U33JUCE4QFPT54RRVY57EF64QS7C4TUO7MWOOGZFAIHBZQUZFB3A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189183712 (12h25m45.933240578s) [noise=true , address=64.49.13.26:4945 , piece_id=BODENY6IN5IS2I2RJ3J3EV3XYKZD4B4RR4PQNKSEXYXVUVYPMC6A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189352560 (12h25m45.933409422s) [noise=true , address=104.36.82.56:4233 , piece_id=ING27ZVSXOL6ASCWIVGSF2KESQ4N6O4AXGQ3HF5EZK5TZ5T3SNNQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189640677 (12h25m45.93369755s) [noise=true , address=104.36.82.20:4461 , piece_id=XGB6IA7NPLFF7XKYATSOG54O7U36IX6BVHVWGA53T7MNVBFM5R2A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189855650 (12h25m45.933912517s) [noise=true , address=104.36.82.62:4439 , piece_id=TQYPKEGJFIZ7D26GWFTFSU4PU75J5DX24QKUEEUUX66EINYJEMIQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190187162 (12h25m45.934244033s) [noise=true , address=104.36.82.13:4201 , piece_id=PD7C77UABKPFT2WM7BOTNOGQV22DUPGWWOGZ5KUUMUVMWSSSHTJA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190334028 (12h25m45.934390898s) [noise=true , address=104.36.82.47:4534 , piece_id=JS7S2OPLDOU5C52ZACTR77R2OVC56PBN6BDNGAPRBNN3UHXQPX3Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190661949 (12h25m45.934718816s) [noise=true , address=77.90.31.16:4742 , piece_id=5USKIHBGUAV6G3BX5BHC6LOEEHDNAJUFK45RWH6OC25QGOMH7V4Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.754845434 (12h25m46.498902284s) [upload=12820]: Next returned nil segment Aug 7 22:05:24.754862619 (12h25m46.49891948s) [upload=12820]: Waiting for error group managing segments... Aug 7 22:05:24.937656798 (12h25m46.681713649s) [noise=true , address=45.91.116.6:4982 , piece_id=LAWUMX2HZNNHRQ3FUODV65COBDIMOZN2HBCSPD42P5DEC4N4VABQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:24.980443483 (12h25m46.724500333s) [noise=true , address=104.36.82.12:4174 , piece_id=DCAYU4CKMHLHOHAN7MKLV5KJTRCCC2IOWSPFPWEMAU3FSIVIXRQA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:24.995491860 (12h25m46.739551452s) [noise=true , address=77.90.31.16:4742 , piece_id=5USKIHBGUAV6G3BX5BHC6LOEEHDNAJUFK45RWH6OC25QGOMH7V4Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.083772095 (12h25m46.827828964s) [noise=true , address=104.36.82.46:4496 , piece_id=XNRPMJSQ3BNIILAQDMRYLFM623WHRZ63ND4RDTHJYLSWWOPNJGHA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.147179204 (12h25m46.891236066s) [noise=true , address=104.36.82.13:4201 , piece_id=PD7C77UABKPFT2WM7BOTNOGQV22DUPGWWOGZ5KUUMUVMWSSSHTJA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.274393636 (12h25m47.018450492s) [noise=true , address=104.36.82.62:4439 , piece_id=TQYPKEGJFIZ7D26GWFTFSU4PU75J5DX24QKUEEUUX66EINYJEMIQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.452784464 (12h25m47.196841314s) [noise=true , address=64.49.13.169:4705 , piece_id=6C45BITH3QWBJ22PLPILYA6LOUQ2JF2D2LOAL2BVJQQWXSXQ37IQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.474459740 (12h25m47.218516586s) [noise=true , address=64.49.13.26:4945 , piece_id=BODENY6IN5IS2I2RJ3J3EV3XYKZD4B4RR4PQNKSEXYXVUVYPMC6A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.481247102 (12h25m47.225303949s) [noise=true , address=64.49.13.140:4515 , piece_id=WQSSOLYXPIBGHNVEG5IWDPLM45QDWPATVU73OJJTRU7NJRQCHDKA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.666066275 (12h25m47.410123138s) [noise=true , address=104.36.82.58:4292 , piece_id=RJJS3MVCC3B3RTAXEM3XLHRSWZEUKIJUSILSKJKT5LJCWNEHX5GA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.666429351 (12h25m47.410486202s) [noise=true , address=104.36.82.56:4233 , piece_id=ING27ZVSXOL6ASCWIVGSF2KESQ4N6O4AXGQ3HF5EZK5TZ5T3SNNQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.782101005 (12h25m47.526157853s) [noise=true , address=104.36.82.16:4314 , piece_id=KX4ZY5LSNB6YBYAS4NKW2QQXXQZV42VGR2TRRKZIL6ZIO3M7BAQQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.863469768 (12h25m47.607526622s) [noise=true , address=104.36.82.43:4868 , piece_id=QSDRGPBV3KWFHHKWJWKLJ3N5APRHOI66TTWAWNOJDCRXQU2P7PAA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.865162168 (12h25m47.609219032s) [noise=true , address=104.36.82.60:4346 , piece_id=CJPTIG7EI264SA2JCH5326GTL33YW6ASGJPXO5ZRWRDD3L5BQTYA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.865201343 (12h25m47.609258205s) [seg_pos={0 0}, upload=12820]: Segment reached optimal threshold of 14 pieces. Aug 7 22:05:25.865362755 (12h25m47.609419601s) [noise=true , address=104.36.82.47:4534 , piece_id=JS7S2OPLDOU5C52ZACTR77R2OVC56PBN6BDNGAPRBNN3UHXQPX3Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VDrNGcCnaKEmG5NN4nex1JoZAgdCEfX5CdsKbxytfLfKzPbr): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865412734 (12h25m47.609469599s) [noise=true , address=104.36.82.20:4461 , piece_id=XGB6IA7NPLFF7XKYATSOG54O7U36IX6BVHVWGA53T7MNVBFM5R2A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nhwT2THUu6ptumqU9Bz1Nd34FHLewx9vWjGFC34j3Lz3RWfkw): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865468394 (12h25m47.609525241s) [noise=true , address=104.36.82.5:4862 , piece_id=W6AHEAV5E5RQR3YQ3LFOO33URBFGYOVTT23S5JKPP7MDNIRFHV5Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865629517 (12h25m47.609686385s) [noise=true , address=104.36.82.22:4530 , piece_id=U33JUCE4QFPT54RRVY57EF64QS7C4TUO7MWOOGZFAIHBZQUZFB3A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1akHUSoQFafzkGTJ385iYaowc87pJBDNfU5hodSCUDFa5ojN3v): protocol: write tcp 139.59.144.22:45386->104.36.82.22:4530: use of closed network connection; context canceled Aug 7 22:05:25.865657252 (12h25m47.609714121s) [seg_pos={0 0}, upload=12820]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:05:25.865814825 (12h25m47.609871688s) [upload=12820]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:05:25.865822925 (12h25m47.609879795s) [upload=12820]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:26.138690358 (12h25m47.882747231s) [upload=12820]: Done uploading segments... Aug 7 22:05:26.138713087 (12h25m47.882769948s) [upload=12820]: Upload finished. err: <nil> Duration: 3.525409192s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [X.X.X..................................................X....] duration: 282.275461ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:18:51 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 282.310783ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 21 out of 60 is failed (threshold=3) [XXXXXXXXXXXXX......X................................X.XXXXXX] duration: 1m0.000875552s FAIL (2025-08-07 22:18:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0.184229810 (12h29m21.928286679s) [upload=12882]: Done starting upload Aug 7 22:09:00.184332233 (12h29m21.928389079s) [upload=12882]: Uploading segments... Aug 7 22:09:00.184404155 (12h29m21.928461026s) [upload=12882]: Got next segment. Inline: true Aug 7 22:09:00.184410758 (12h29m21.928467627s) [upload=12882]: Waiting for error group managing segments... Aug 7 22:09:00.184480435 (12h29m21.928537304s) [upload=12882]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:09:00.184487151 (12h29m21.928544217s) [upload=12882]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:09:00.184491624 (12h29m21.928548491s) [upload=12882]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:09:00.583850980 (12h29m22.327907833s) [upload=12882]: Done uploading segments... Aug 7 22:09:00.583888682 (12h29m22.327945555s) [upload=12882]: Upload finished. err: <nil> Aug 7 22:09:40.429348924 (12h30m2.17340578s) [upload=12891]: Starting upload Aug 7 22:09:40.429449124 (12h30m2.173505984s) [upload=12891]: Done starting upload Aug 7 22:09:40.429576700 (12h30m2.173633561s) [upload=12891]: Uploading segments... Aug 7 22:09:40.429631265 (12h30m2.173688133s) [upload=12891]: Got next segment. Inline: true Aug 7 22:09:40.429637613 (12h30m2.173694464s) [upload=12891]: Waiting for error group managing segments... Aug 7 22:09:40.429689508 (12h30m2.173746362s) [upload=12891]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:09:40.429698290 (12h30m2.17375514s) [upload=12891]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:09:40.429702162 (12h30m2.173759014s) [upload=12891]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:09:40.614648207 (12h30m2.358705075s) [upload=12891]: Done uploading segments... Aug 7 22:09:40.614672259 (12h30m2.358729124s) [upload=12891]: Upload finished. err: <nil> Aug 7 22:10:28.314751006 (12h30m50.058807867s) [upload=12903]: Starting upload Aug 7 22:10:28.314813386 (12h30m50.058870247s) [upload=12903]: Done starting upload Aug 7 22:10:28.314903941 (12h30m50.058960802s) [upload=12903]: Uploading segments... Aug 7 22:10:28.314992884 (12h30m50.059049757s) [upload=12903]: Got next segment. Inline: true Aug 7 22:10:28.315015785 (12h30m50.059072647s) [upload=12903]: Waiting for error group managing segments... Aug 7 22:10:28.315072549 (12h30m50.059129415s) [upload=12903]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:10:28.315079820 (12h30m50.059136685s) [upload=12903]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:10:28.315099592 (12h30m50.059156457s) [upload=12903]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:11:28.039892681 (12h31m49.783949531s) [upload=12903]: Done uploading segments... Aug 7 22:11:28.039913432 (12h31m49.783970283s) [upload=12903]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:11:35.297516727 (12h31m57.041573588s) [upload=12923]: Starting upload Aug 7 22:11:35.297601596 (12h31m57.041658469s) [upload=12923]: Done starting upload Aug 7 22:11:35.297686234 (12h31m57.041743086s) [upload=12923]: Uploading segments... Aug 7 22:11:35.297779680 (12h31m57.041836543s) [upload=12923]: Got next segment. Inline: true Aug 7 22:11:35.297790270 (12h31m57.041847121s) [upload=12923]: Waiting for error group managing segments... Aug 7 22:11:35.297854321 (12h31m57.041911182s) [upload=12923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:11:35.297860387 (12h31m57.041917249s) [upload=12923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:11:35.297863885 (12h31m57.041920745s) [upload=12923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:11:44.558966532 (12h32m6.303023384s) [upload=12923]: Done uploading segments... Aug 7 22:11:44.558985310 (12h32m6.303042163s) [upload=12923]: Upload finished. err: <nil> Aug 7 22:12:36.559571434 (12h32m58.30362831s) [upload=12938]: Starting upload Aug 7 22:12:36.559699331 (12h32m58.3037562s) [upload=12938]: Done starting upload Aug 7 22:12:36.559803583 (12h32m58.30386044s) [upload=12938]: Uploading segments... Aug 7 22:12:36.560034082 (12h32m58.304090945s) [upload=12938]: Got next segment. Inline: true Aug 7 22:12:36.560041392 (12h32m58.304098252s) [upload=12938]: Waiting for error group managing segments... Aug 7 22:12:36.560151251 (12h32m58.304208113s) [upload=12938]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:12:36.560159993 (12h32m58.304216852s) [upload=12938]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:12:36.560164091 (12h32m58.304220952s) [upload=12938]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:13:36.275827133 (12h33m58.019883992s) [upload=12938]: Done uploading segments... Aug 7 22:13:36.275867976 (12h33m58.019924837s) [upload=12938]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:13:36.703916041 (12h33m58.447972892s) [upload=12954]: Starting upload Aug 7 22:13:36.704016678 (12h33m58.448073543s) [upload=12954]: Done starting upload Aug 7 22:13:36.704126028 (12h33m58.448182877s) [upload=12954]: Uploading segments... Aug 7 22:13:36.704202003 (12h33m58.448258875s) [upload=12954]: Got next segment. Inline: true Aug 7 22:13:36.704208724 (12h33m58.448265583s) [upload=12954]: Waiting for error group managing segments... Aug 7 22:13:36.704253077 (12h33m58.44830994s) [upload=12954]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:13:36.704259505 (12h33m58.448316353s) [upload=12954]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:13:36.704263329 (12h33m58.448320184s) [upload=12954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:14:36.426520849 (12h34m58.170577716s) [upload=12954]: Done uploading segments... Aug 7 22:14:36.426547324 (12h34m58.170604188s) [upload=12954]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:14:36.851362491 (12h34m58.595419352s) [upload=12969]: Starting upload Aug 7 22:14:36.851438058 (12h34m58.595494919s) [upload=12969]: Done starting upload Aug 7 22:14:36.851531336 (12h34m58.595588182s) [upload=12969]: Uploading segments... Aug 7 22:14:36.851620028 (12h34m58.595676898s) [upload=12969]: Got next segment. Inline: true Aug 7 22:14:36.851627191 (12h34m58.595684056s) [upload=12969]: Waiting for error group managing segments... Aug 7 22:14:36.851698724 (12h34m58.595755585s) [upload=12969]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:14:36.851706200 (12h34m58.595763081s) [upload=12969]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:14:36.851710849 (12h34m58.595767717s) [upload=12969]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:15:36.582802839 (12h35m58.326859693s) [upload=12969]: Done uploading segments... Aug 7 22:15:36.582825807 (12h35m58.326882658s) [upload=12969]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:37.013505363 (12h35m58.757562231s) [upload=12985]: Starting upload Aug 7 22:15:37.013605304 (12h35m58.757662168s) [upload=12985]: Done starting upload Aug 7 22:15:37.013715315 (12h35m58.757772164s) [upload=12985]: Uploading segments... Aug 7 22:15:37.013761212 (12h35m58.757818081s) [upload=12985]: Got next segment. Inline: true Aug 7 22:15:37.013776983 (12h35m58.757833843s) [upload=12985]: Waiting for error group managing segments... Aug 7 22:15:37.013820691 (12h35m58.757877552s) [upload=12985]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:15:37.013826603 (12h35m58.757883467s) [upload=12985]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:15:37.013830513 (12h35m58.757887374s) [upload=12985]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:16:36.733288222 (12h36m58.477345085s) [upload=12985]: Done uploading segments... Aug 7 22:16:36.733314893 (12h36m58.477371754s) [upload=12985]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:16:37.187949023 (12h36m58.932005874s) [upload=13003]: Starting upload Aug 7 22:16:37.188024595 (12h36m58.932081456s) [upload=13003]: Done starting upload Aug 7 22:16:37.188115827 (12h36m58.932172677s) [upload=13003]: Uploading segments... Aug 7 22:16:37.188173315 (12h36m58.932230182s) [upload=13003]: Got next segment. Inline: true Aug 7 22:16:37.188181498 (12h36m58.932238365s) [upload=13003]: Waiting for error group managing segments... Aug 7 22:16:37.188252566 (12h36m58.932309427s) [upload=13003]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:16:37.188260045 (12h36m58.932316913s) [upload=13003]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:16:37.188263607 (12h36m58.932320469s) [upload=13003]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:17:36.907306861 (12h37m58.651363739s) [upload=13003]: Done uploading segments... Aug 7 22:17:36.907340408 (12h37m58.651397275s) [upload=13003]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:46.749223731 (12h38m8.493280581s) [upload=13019]: Starting upload Aug 7 22:17:46.749288163 (12h38m8.493345017s) [upload=13019]: Done starting upload Aug 7 22:17:46.749381044 (12h38m8.493437894s) [upload=13019]: Uploading segments... Aug 7 22:17:46.749423663 (12h38m8.493480532s) [upload=13019]: Got next segment. Inline: true Aug 7 22:17:46.749428712 (12h38m8.493485573s) [upload=13019]: Waiting for error group managing segments... Aug 7 22:17:46.749471140 (12h38m8.49352799s) [upload=13019]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:17:46.749476259 (12h38m8.493533119s) [upload=13019]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:46.749479952 (12h38m8.493536802s) [upload=13019]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:18:46.471549080 (12h39m8.215605941s) [upload=13019]: Done uploading segments... Aug 7 22:18:46.471571547 (12h39m8.215628407s) [upload=13019]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.000901395s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 110.612222ms remote: "tcp:185.244.226.2:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-07 22:18:46 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 110.658554ms edge linksharing bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 17 out of 60 is failed (threshold=3) [XXXXXXXXX.X..........................................XXXXXXX] duration: 30.00059816s FAIL (2025-08-07 22:18:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000612715s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 24 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXX....................................XXXXXXXXX] duration: 30.00095551s FAIL (2025-08-07 22:18:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d Aug 7 22:10:21.179644873 (12h30m42.923701721s) [upload=12901]: Uploading segments... Aug 7 22:10:21.179710572 (12h30m42.923767442s) [upload=12901]: Got next segment. Inline: true Aug 7 22:10:21.179717046 (12h30m42.923773918s) [upload=12901]: Waiting for error group managing segments... Aug 7 22:10:21.179768868 (12h30m42.923825739s) [upload=12901]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:10:21.179774459 (12h30m42.923831332s) [upload=12901]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:10:21.179778016 (12h30m42.923834886s) [upload=12901]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:10:21.324404712 (12h30m43.068461582s) [upload=12901]: Done uploading segments... Aug 7 22:10:21.324445701 (12h30m43.068502568s) [upload=12901]: Upload finished. err: <nil> Aug 7 22:11:27.454578904 (12h31m49.198635753s) [upload=12920]: Starting upload Aug 7 22:11:27.454695442 (12h31m49.19875231s) [upload=12920]: Done starting upload Aug 7 22:11:27.454814832 (12h31m49.198871679s) [upload=12920]: Uploading segments... Aug 7 22:11:27.454893840 (12h31m49.198950813s) [upload=12920]: Got next segment. Inline: true Aug 7 22:11:27.454915725 (12h31m49.198972587s) [upload=12920]: Waiting for error group managing segments... Aug 7 22:11:27.454970562 (12h31m49.199027411s) [upload=12920]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:11:27.454977263 (12h31m49.199034131s) [upload=12920]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:11:27.455074233 (12h31m49.199131094s) [upload=12920]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:11:57.179550579 (12h32m18.92360743s) [upload=12920]: Done uploading segments... Aug 7 22:11:57.179569786 (12h32m18.923626636s) [upload=12920]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:12:19.183043096 (12h32m40.927099946s) [upload=12933]: Starting upload Aug 7 22:12:19.183127401 (12h32m40.927184261s) [upload=12933]: Done starting upload Aug 7 22:12:19.183328044 (12h32m40.92738489s) [upload=12933]: Uploading segments... Aug 7 22:12:19.183400189 (12h32m40.92745706s) [upload=12933]: Got next segment. Inline: true Aug 7 22:12:19.183407329 (12h32m40.927464177s) [upload=12933]: Waiting for error group managing segments... Aug 7 22:12:19.183479311 (12h32m40.927536158s) [upload=12933]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:12:19.183484643 (12h32m40.927541506s) [upload=12933]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:12:19.183487379 (12h32m40.927544225s) [upload=12933]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:12:48.909423390 (12h33m10.653480257s) [upload=12933]: Done uploading segments... Aug 7 22:12:48.909444894 (12h33m10.653501768s) [upload=12933]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:13:10.831892676 (12h33m32.575949525s) [upload=12948]: Starting upload Aug 7 22:13:10.831993207 (12h33m32.576050068s) [upload=12948]: Done starting upload Aug 7 22:13:10.832071605 (12h33m32.576128457s) [upload=12948]: Uploading segments... Aug 7 22:13:10.832146431 (12h33m32.576203291s) [upload=12948]: Got next segment. Inline: true Aug 7 22:13:10.832152814 (12h33m32.576209674s) [upload=12948]: Waiting for error group managing segments... Aug 7 22:13:10.832208966 (12h33m32.576265827s) [upload=12948]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:13:10.832214384 (12h33m32.576271244s) [upload=12948]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:13:10.832217904 (12h33m32.576274764s) [upload=12948]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:13:40.559928781 (12h34m2.303985629s) [upload=12948]: Done uploading segments... Aug 7 22:13:40.559950271 (12h34m2.304007117s) [upload=12948]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:13:48.685772427 (12h34m10.429829287s) [upload=12958]: Starting upload Aug 7 22:13:48.685858893 (12h34m10.429915753s) [upload=12958]: Done starting upload Aug 7 22:13:48.685982193 (12h34m10.430039042s) [upload=12958]: Uploading segments... Aug 7 22:13:48.686047966 (12h34m10.430104835s) [upload=12958]: Got next segment. Inline: true Aug 7 22:13:48.686053563 (12h34m10.430110423s) [upload=12958]: Waiting for error group managing segments... Aug 7 22:13:48.686096300 (12h34m10.43015316s) [upload=12958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:13:48.686101341 (12h34m10.430158201s) [upload=12958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:13:48.686104243 (12h34m10.430161104s) [upload=12958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:14:18.399606594 (12h34m40.143663447s) [upload=12958]: Done uploading segments... Aug 7 22:14:18.399627822 (12h34m40.143684675s) [upload=12958]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:14:59.902619970 (12h35m21.646676816s) [upload=12977]: Starting upload Aug 7 22:14:59.902685895 (12h35m21.646742756s) [upload=12977]: Done starting upload Aug 7 22:14:59.902751694 (12h35m21.646808545s) [upload=12977]: Uploading segments... Aug 7 22:14:59.902795346 (12h35m21.646852214s) [upload=12977]: Got next segment. Inline: true Aug 7 22:14:59.902801495 (12h35m21.646858346s) [upload=12977]: Waiting for error group managing segments... Aug 7 22:14:59.902858709 (12h35m21.64691556s) [upload=12977]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:14:59.902863962 (12h35m21.646920811s) [upload=12977]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:14:59.902867458 (12h35m21.646924309s) [upload=12977]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:15:29.636302762 (12h35m51.380359612s) [upload=12977]: Done uploading segments... Aug 7 22:15:29.636325398 (12h35m51.380382249s) [upload=12977]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:45.428850713 (12h36m7.172907563s) [upload=12988]: Starting upload Aug 7 22:15:45.428978829 (12h36m7.173035696s) [upload=12988]: Done starting upload Aug 7 22:15:45.429103446 (12h36m7.173160298s) [upload=12988]: Uploading segments... Aug 7 22:15:45.429150754 (12h36m7.173207623s) [upload=12988]: Got next segment. Inline: true Aug 7 22:15:45.429156339 (12h36m7.173213199s) [upload=12988]: Waiting for error group managing segments... Aug 7 22:15:45.429230361 (12h36m7.173287221s) [upload=12988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:15:45.429247463 (12h36m7.173304325s) [upload=12988]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:15:45.429251133 (12h36m7.173307993s) [upload=12988]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:16:15.155440875 (12h36m36.899497725s) [upload=12988]: Done uploading segments... Aug 7 22:16:15.155471506 (12h36m36.899528367s) [upload=12988]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:16:15.598565945 (12h36m37.342622791s) [upload=12999]: Starting upload Aug 7 22:16:15.598645050 (12h36m37.342701913s) [upload=12999]: Done starting upload Aug 7 22:16:15.598718650 (12h36m37.342775497s) [upload=12999]: Uploading segments... Aug 7 22:16:15.598774264 (12h36m37.34283113s) [upload=12999]: Got next segment. Inline: true Aug 7 22:16:15.598779124 (12h36m37.342835984s) [upload=12999]: Waiting for error group managing segments... Aug 7 22:16:15.598830461 (12h36m37.342887324s) [upload=12999]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:16:15.598854102 (12h36m37.342910963s) [upload=12999]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:16:15.598857833 (12h36m37.342914694s) [upload=12999]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:16:45.328584439 (12h37m7.072641289s) [upload=12999]: Done uploading segments... Aug 7 22:16:45.328609005 (12h37m7.072665856s) [upload=12999]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:27.480756310 (12h37m49.22481316s) [upload=13015]: Starting upload Aug 7 22:17:27.480859326 (12h37m49.22491619s) [upload=13015]: Done starting upload Aug 7 22:17:27.480944450 (12h37m49.225001301s) [upload=13015]: Uploading segments... Aug 7 22:17:27.481000891 (12h37m49.225057757s) [upload=13015]: Got next segment. Inline: true Aug 7 22:17:27.481006468 (12h37m49.225063328s) [upload=13015]: Waiting for error group managing segments... Aug 7 22:17:27.481069727 (12h37m49.225126586s) [upload=13015]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:17:27.481076622 (12h37m49.225133484s) [upload=13015]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:27.481081288 (12h37m49.225138155s) [upload=13015]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:17:57.211870572 (12h38m18.955927423s) [upload=13015]: Done uploading segments... Aug 7 22:17:57.211892595 (12h38m18.955949444s) [upload=13015]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:14.331617780 (12h38m36.07567463s) [upload=13026]: Starting upload Aug 7 22:18:14.331690520 (12h38m36.07574737s) [upload=13026]: Done starting upload Aug 7 22:18:14.331789874 (12h38m36.075846724s) [upload=13026]: Uploading segments... Aug 7 22:18:14.331829497 (12h38m36.075886363s) [upload=13026]: Got next segment. Inline: true Aug 7 22:18:14.331834588 (12h38m36.075891448s) [upload=13026]: Waiting for error group managing segments... Aug 7 22:18:14.331890028 (12h38m36.075946878s) [upload=13026]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:18:14.331895594 (12h38m36.075952444s) [upload=13026]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:14.331899378 (12h38m36.075956238s) [upload=13026]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:18:44.050649404 (12h39m5.794706254s) [upload=13026]: Done uploading segments... Aug 7 22:18:44.050671176 (12h39m5.794728026s) [upload=13026]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000994492s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 22 out of 60 is failed (threshold=3) [XXXXXXXXX..X.XX..................................X.XXXXXXXXX] duration: 30.001656857s FAIL (2025-08-07 22:18:41 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001674024s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 26 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXX......X.X...........................XXXXXXX] duration: 1m0.001026581s FAIL (2025-08-07 22:18:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001090621s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X..........X.....X.........X..X...........X................] duration: 4.093677918s FAIL (2025-08-07 22:18:21 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 5.727759364s) [upload=12820]: Starting upload Aug 7 22:05:23.983754833 (12h25m45.727811679s) [upload=12820]: Done starting upload Aug 7 22:05:23.983988533 (12h25m45.728045379s) [upload=12820]: Uploading segments... Aug 7 22:05:23.984022348 (12h25m45.728079214s) [upload=12820]: Got next segment. Inline: false Aug 7 22:05:23.984055700 (12h25m45.728112548s) [upload=12820]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:23.984060538 (12h25m45.728117398s) [upload=12820]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:05:24.185392691 (12h25m45.929449561s) [seg_pos={0 0}, upload=12820]: Begin upload segment... Aug 7 22:05:24.185557956 (12h25m45.929614817s) [seg_pos={0 0}, upload=12820]: Done begin upload segment. Aug 7 22:05:24.185603301 (12h25m45.929660177s) [noise=true , address=64.49.13.169:4705 , piece_id=6C45BITH3QWBJ22PLPILYA6LOUQ2JF2D2LOAL2BVJQQWXSXQ37IQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186037040 (12h25m45.930093906s) [noise=true , address=104.36.82.12:4174 , piece_id=DCAYU4CKMHLHOHAN7MKLV5KJTRCCC2IOWSPFPWEMAU3FSIVIXRQA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186570414 (12h25m45.930627287s) [noise=true , address=104.36.82.16:4314 , piece_id=KX4ZY5LSNB6YBYAS4NKW2QQXXQZV42VGR2TRRKZIL6ZIO3M7BAQQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186824559 (12h25m45.930881425s) [noise=true , address=104.36.82.46:4496 , piece_id=XNRPMJSQ3BNIILAQDMRYLFM623WHRZ63ND4RDTHJYLSWWOPNJGHA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187208122 (12h25m45.931264991s) [noise=true , address=104.36.82.43:4868 , piece_id=QSDRGPBV3KWFHHKWJWKLJ3N5APRHOI66TTWAWNOJDCRXQU2P7PAA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187658761 (12h25m45.931715627s) [noise=true , address=104.36.82.60:4346 , piece_id=CJPTIG7EI264SA2JCH5326GTL33YW6ASGJPXO5ZRWRDD3L5BQTYA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187940002 (12h25m45.931996872s) [noise=true , address=64.49.13.140:4515 , piece_id=WQSSOLYXPIBGHNVEG5IWDPLM45QDWPATVU73OJJTRU7NJRQCHDKA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188116203 (12h25m45.932173069s) [noise=true , address=45.91.116.6:4982 , piece_id=LAWUMX2HZNNHRQ3FUODV65COBDIMOZN2HBCSPD42P5DEC4N4VABQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188531115 (12h25m45.932587976s) [noise=true , address=104.36.82.58:4292 , piece_id=RJJS3MVCC3B3RTAXEM3XLHRSWZEUKIJUSILSKJKT5LJCWNEHX5GA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188637273 (12h25m45.932694145s) [noise=true , address=104.36.82.5:4862 , piece_id=W6AHEAV5E5RQR3YQ3LFOO33URBFGYOVTT23S5JKPP7MDNIRFHV5Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188961006 (12h25m45.933017872s) [noise=true , address=104.36.82.22:4530 , piece_id=U33JUCE4QFPT54RRVY57EF64QS7C4TUO7MWOOGZFAIHBZQUZFB3A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189183712 (12h25m45.933240578s) [noise=true , address=64.49.13.26:4945 , piece_id=BODENY6IN5IS2I2RJ3J3EV3XYKZD4B4RR4PQNKSEXYXVUVYPMC6A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189352560 (12h25m45.933409422s) [noise=true , address=104.36.82.56:4233 , piece_id=ING27ZVSXOL6ASCWIVGSF2KESQ4N6O4AXGQ3HF5EZK5TZ5T3SNNQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189640677 (12h25m45.93369755s) [noise=true , address=104.36.82.20:4461 , piece_id=XGB6IA7NPLFF7XKYATSOG54O7U36IX6BVHVWGA53T7MNVBFM5R2A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189855650 (12h25m45.933912517s) [noise=true , address=104.36.82.62:4439 , piece_id=TQYPKEGJFIZ7D26GWFTFSU4PU75J5DX24QKUEEUUX66EINYJEMIQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190187162 (12h25m45.934244033s) [noise=true , address=104.36.82.13:4201 , piece_id=PD7C77UABKPFT2WM7BOTNOGQV22DUPGWWOGZ5KUUMUVMWSSSHTJA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190334028 (12h25m45.934390898s) [noise=true , address=104.36.82.47:4534 , piece_id=JS7S2OPLDOU5C52ZACTR77R2OVC56PBN6BDNGAPRBNN3UHXQPX3Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190661949 (12h25m45.934718816s) [noise=true , address=77.90.31.16:4742 , piece_id=5USKIHBGUAV6G3BX5BHC6LOEEHDNAJUFK45RWH6OC25QGOMH7V4Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.754845434 (12h25m46.498902284s) [upload=12820]: Next returned nil segment Aug 7 22:05:24.754862619 (12h25m46.49891948s) [upload=12820]: Waiting for error group managing segments... Aug 7 22:05:24.937656798 (12h25m46.681713649s) [noise=true , address=45.91.116.6:4982 , piece_id=LAWUMX2HZNNHRQ3FUODV65COBDIMOZN2HBCSPD42P5DEC4N4VABQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:24.980443483 (12h25m46.724500333s) [noise=true , address=104.36.82.12:4174 , piece_id=DCAYU4CKMHLHOHAN7MKLV5KJTRCCC2IOWSPFPWEMAU3FSIVIXRQA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:24.995491860 (12h25m46.739551452s) [noise=true , address=77.90.31.16:4742 , piece_id=5USKIHBGUAV6G3BX5BHC6LOEEHDNAJUFK45RWH6OC25QGOMH7V4Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.083772095 (12h25m46.827828964s) [noise=true , address=104.36.82.46:4496 , piece_id=XNRPMJSQ3BNIILAQDMRYLFM623WHRZ63ND4RDTHJYLSWWOPNJGHA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.147179204 (12h25m46.891236066s) [noise=true , address=104.36.82.13:4201 , piece_id=PD7C77UABKPFT2WM7BOTNOGQV22DUPGWWOGZ5KUUMUVMWSSSHTJA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.274393636 (12h25m47.018450492s) [noise=true , address=104.36.82.62:4439 , piece_id=TQYPKEGJFIZ7D26GWFTFSU4PU75J5DX24QKUEEUUX66EINYJEMIQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.452784464 (12h25m47.196841314s) [noise=true , address=64.49.13.169:4705 , piece_id=6C45BITH3QWBJ22PLPILYA6LOUQ2JF2D2LOAL2BVJQQWXSXQ37IQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.474459740 (12h25m47.218516586s) [noise=true , address=64.49.13.26:4945 , piece_id=BODENY6IN5IS2I2RJ3J3EV3XYKZD4B4RR4PQNKSEXYXVUVYPMC6A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.481247102 (12h25m47.225303949s) [noise=true , address=64.49.13.140:4515 , piece_id=WQSSOLYXPIBGHNVEG5IWDPLM45QDWPATVU73OJJTRU7NJRQCHDKA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.666066275 (12h25m47.410123138s) [noise=true , address=104.36.82.58:4292 , piece_id=RJJS3MVCC3B3RTAXEM3XLHRSWZEUKIJUSILSKJKT5LJCWNEHX5GA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.666429351 (12h25m47.410486202s) [noise=true , address=104.36.82.56:4233 , piece_id=ING27ZVSXOL6ASCWIVGSF2KESQ4N6O4AXGQ3HF5EZK5TZ5T3SNNQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.782101005 (12h25m47.526157853s) [noise=true , address=104.36.82.16:4314 , piece_id=KX4ZY5LSNB6YBYAS4NKW2QQXXQZV42VGR2TRRKZIL6ZIO3M7BAQQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.863469768 (12h25m47.607526622s) [noise=true , address=104.36.82.43:4868 , piece_id=QSDRGPBV3KWFHHKWJWKLJ3N5APRHOI66TTWAWNOJDCRXQU2P7PAA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.865162168 (12h25m47.609219032s) [noise=true , address=104.36.82.60:4346 , piece_id=CJPTIG7EI264SA2JCH5326GTL33YW6ASGJPXO5ZRWRDD3L5BQTYA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.865201343 (12h25m47.609258205s) [seg_pos={0 0}, upload=12820]: Segment reached optimal threshold of 14 pieces. Aug 7 22:05:25.865362755 (12h25m47.609419601s) [noise=true , address=104.36.82.47:4534 , piece_id=JS7S2OPLDOU5C52ZACTR77R2OVC56PBN6BDNGAPRBNN3UHXQPX3Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VDrNGcCnaKEmG5NN4nex1JoZAgdCEfX5CdsKbxytfLfKzPbr): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865412734 (12h25m47.609469599s) [noise=true , address=104.36.82.20:4461 , piece_id=XGB6IA7NPLFF7XKYATSOG54O7U36IX6BVHVWGA53T7MNVBFM5R2A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nhwT2THUu6ptumqU9Bz1Nd34FHLewx9vWjGFC34j3Lz3RWfkw): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865468394 (12h25m47.609525241s) [noise=true , address=104.36.82.5:4862 , piece_id=W6AHEAV5E5RQR3YQ3LFOO33URBFGYOVTT23S5JKPP7MDNIRFHV5Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865629517 (12h25m47.609686385s) [noise=true , address=104.36.82.22:4530 , piece_id=U33JUCE4QFPT54RRVY57EF64QS7C4TUO7MWOOGZFAIHBZQUZFB3A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1akHUSoQFafzkGTJ385iYaowc87pJBDNfU5hodSCUDFa5ojN3v): protocol: write tcp 139.59.144.22:45386->104.36.82.22:4530: use of closed network connection; context canceled Aug 7 22:05:25.865657252 (12h25m47.609714121s) [seg_pos={0 0}, upload=12820]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:05:25.865814825 (12h25m47.609871688s) [upload=12820]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:05:25.865822925 (12h25m47.609879795s) [upload=12820]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:26.138690358 (12h25m47.882747231s) [upload=12820]: Done uploading segments... Aug 7 22:05:26.138713087 (12h25m47.882769948s) [upload=12820]: Upload finished. err: <nil> Duration: 4.093711642s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 17 out of 60 is failed (threshold=3) [XXXXXXXXXX.X..........................................XXXXXX] duration: 30.001151864s FAIL (2025-08-07 22:18:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001180191s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [......................XXXX..............................XXXX] duration: 1m0.000701673s FAIL (2025-08-07 22:18:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 1.116.13:4166 , piece_id=HSFXWLME6GIFAWHD73GW4H6CBPBJZ4MPTB7JD7XY2YBTZESICNRA, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.395528042 (12h32m18.139584909s) [noise=true , address=104.36.82.60:4343 , piece_id=A5ZQY7FYEN7BVIGGB6T43UTFNOC6NGPBM3XJ23QSLVRFFVSVYJCQ, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.395636576 (12h32m18.139693442s) [noise=true , address=104.36.82.13:4188 , piece_id=C24Q324TD3OIGJSBOKVUN2VH5ORNKINWDHBTA3O56VSRPJN52SGA, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.395864366 (12h32m18.139921236s) [noise=true , address=104.36.82.47:4520 , piece_id=DLEDG3JMSGGGFQFGNJWV4XIDPXITG54XCYTRXXJIRR4DOXWIDAVA, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.396030181 (12h32m18.140087045s) [noise=true , address=104.36.82.36:4645 , piece_id=P63BF2T4FLUK7DMM4GQY277QTDGZXT3CD4Z5GFDMUQTTJN77TZIQ, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.396205503 (12h32m18.140262364s) [noise=true , address=45.91.116.9:4272 , piece_id=3QGHJZTVECLQTJGC46GZ5276LVTHR36LBTK5Q5FSRGD7QXOUJJ6A, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.397743950 (12h32m18.141800804s) [noise=true , address=64.49.13.139:4471 , piece_id=QTORROJDLNZQTDC2VFU3Q5DW47TA7VM7FZAVKTACLLCPMHGOC52A, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.398091992 (12h32m18.142148846s) [noise=true , address=104.36.82.43:4837 , piece_id=WALP4A6IL6VR6577XHAIATXZGBECNO6BL5Y5Z4I4DMPCNIK7X5QQ, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.979185930 (12h32m18.723242776s) [upload=12925]: Next returned nil segment Aug 7 22:11:56.979206213 (12h32m18.723263074s) [upload=12925]: Waiting for error group managing segments... Aug 7 22:11:57.134173791 (12h32m18.878230638s) [noise=true , address=104.36.82.13:4188 , piece_id=C24Q324TD3OIGJSBOKVUN2VH5ORNKINWDHBTA3O56VSRPJN52SGA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.151552906 (12h32m18.895609777s) [noise=true , address=104.36.82.6:4895 , piece_id=HUPRXOSI2CS2UDOF47BME2BYCCLXANUCJAYVZV4JNRMPQQUDZQFA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.151985548 (12h32m18.896042414s) [noise=true , address=104.36.82.60:4343 , piece_id=A5ZQY7FYEN7BVIGGB6T43UTFNOC6NGPBM3XJ23QSLVRFFVSVYJCQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.177078648 (12h32m18.921135515s) [noise=true , address=104.36.82.34:4566 , piece_id=GR2IDGJL2TB5VYWZOIW2C652I5B5XUUXXI4SIELMP3V4GMO2ZH4Q, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.180968885 (12h32m18.925025749s) [noise=true , address=104.36.82.45:4940 , piece_id=E2BSUMWRGNFOCNBBT32GAMW3JN4PFKNKSYXK5AEUGQQKRK4I6FKA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.182573730 (12h32m18.926630601s) [noise=true , address=104.36.82.43:4837 , piece_id=WALP4A6IL6VR6577XHAIATXZGBECNO6BL5Y5Z4I4DMPCNIK7X5QQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.185217873 (12h32m18.929274727s) [noise=true , address=104.36.82.60:4346 , piece_id=OCI6XALWGEOAF64WRNURL5VVPFWYBT24GXK4GNQMUX2MJYZACROQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.191081154 (12h32m18.935138022s) [noise=true , address=104.36.82.23:4448 , piece_id=RD4YTZPX2A4SRE2XWYS45DWZRG6CUS5VLCYBMJPBJ24LGJTGITFQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.205956300 (12h32m18.950013175s) [noise=true , address=104.36.82.36:4645 , piece_id=P63BF2T4FLUK7DMM4GQY277QTDGZXT3CD4Z5GFDMUQTTJN77TZIQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.214668342 (12h32m18.958725212s) [noise=true , address=104.36.82.39:4733 , piece_id=VZPW75CLRYOWCVQGWBWAOGWKFCXYLYOM6OIDMXRBR5IRGZ7DGLLQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.245950582 (12h32m18.990007453s) [noise=true , address=104.36.82.16:4310 , piece_id=UAVZCLFOGCC6T4HX676IX6OWFNRZRTXL3PITG5IZFR2JATIZS2DQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.280359350 (12h32m19.024416213s) [noise=true , address=45.91.116.9:4272 , piece_id=3QGHJZTVECLQTJGC46GZ5276LVTHR36LBTK5Q5FSRGD7QXOUJJ6A, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.338710901 (12h32m19.082767752s) [noise=true , address=45.91.116.13:4166 , piece_id=HSFXWLME6GIFAWHD73GW4H6CBPBJZ4MPTB7JD7XY2YBTZESICNRA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.617684582 (12h32m19.361741428s) [noise=true , address=104.36.82.62:4439 , piece_id=2P7NHL5WLDREVBGDS4IMRS6D4DZBQHRYFM7USCDK2KU74SKDLW4A, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.617723267 (12h32m19.361780128s) [seg_pos={0 0}, upload=12925]: Segment reached optimal threshold of 14 pieces. Aug 7 22:11:57.617992734 (12h32m19.36204958s) [noise=true , address=104.36.82.47:4520 , piece_id=DLEDG3JMSGGGFQFGNJWV4XIDPXITG54XCYTRXXJIRR4DOXWIDAVA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 22:11:57.618089635 (12h32m19.362146495s) [noise=true , address=64.49.13.137:4498 , piece_id=7SLUBM4U2CKK25FCFPHVKBA4D4TKCRZXKASC7LR2ASBM7C7MDQKQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: expected piece hash; context canceled; EOF Aug 7 22:11:57.618135706 (12h32m19.362192556s) [noise=true , address=104.36.82.5:4855 , piece_id=OEDGYJGQIIJHLEBR5OPPQCO75KXIGPFZBDCCTBYSUV7XQADJRSAA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh): protocol: write tcp 139.59.144.22:57090->104.36.82.5:4855: use of closed network connection; write tcp 139.59.144.22:57090->104.36.82.5:4855: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57090->104.36.82.5:4855: use of closed network connection Aug 7 22:11:57.618165938 (12h32m19.362222785s) [noise=true , address=64.49.13.139:4471 , piece_id=QTORROJDLNZQTDC2VFU3Q5DW47TA7VM7FZAVKTACLLCPMHGOC52A, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129YkXH84aYSubinAC57swW24qCCEtvJTSMDGoxP9WdgpGcZxLr): protocol: expected piece hash; context canceled; EOF Aug 7 22:11:57.618181168 (12h32m19.362238028s) [seg_pos={0 0}, upload=12925]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:11:57.618262981 (12h32m19.362319833s) [upload=12925]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:11:57.618268565 (12h32m19.362325425s) [upload=12925]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:15:11.462898274 (12h35m33.206955137s) [upload=12925]: Done uploading segments... Aug 7 22:15:11.462927061 (12h35m33.206983911s) [upload=12925]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:11.893865947 (12h35m33.637922798s) [upload=12978]: Starting upload Aug 7 22:15:11.894006161 (12h35m33.638063025s) [upload=12978]: Done starting upload Aug 7 22:15:11.894128456 (12h35m33.63818531s) [upload=12978]: Uploading segments... Aug 7 22:15:11.894439225 (12h35m33.638496094s) [upload=12978]: Got next segment. Inline: false Aug 7 22:15:11.894516777 (12h35m33.638573639s) [upload=12978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:15:11.894545833 (12h35m33.638602694s) [upload=12978]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:16:11.626077800 (12h36m33.370134666s) [upload=12978]: Next segment err: context deadline exceeded Aug 7 22:16:11.626165441 (12h36m33.370222301s) [upload=12978]: Done uploading segments... Aug 7 22:16:11.626183066 (12h36m33.370239913s) [upload=12978]: Upload finished. err: context deadline exceeded Aug 7 22:16:12.058909015 (12h36m33.802965875s) [upload=12997]: Starting upload Aug 7 22:16:12.058982372 (12h36m33.803039232s) [upload=12997]: Done starting upload Aug 7 22:16:12.059120653 (12h36m33.803177509s) [upload=12997]: Uploading segments... Aug 7 22:16:12.059593126 (12h36m33.803649985s) [upload=12997]: Got next segment. Inline: false Aug 7 22:16:12.059677172 (12h36m33.803734023s) [upload=12997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:16:12.059684424 (12h36m33.803741287s) [upload=12997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:17:11.787443939 (12h37m33.531500809s) [upload=12997]: Next segment err: context deadline exceeded Aug 7 22:17:11.787498695 (12h37m33.531555556s) [upload=12997]: Done uploading segments... Aug 7 22:17:11.787527052 (12h37m33.531583904s) [upload=12997]: Upload finished. err: context deadline exceeded Aug 7 22:17:12.226578214 (12h37m33.970635065s) [upload=13013]: Starting upload Aug 7 22:17:12.226666960 (12h37m33.970723825s) [upload=13013]: Done starting upload Aug 7 22:17:12.226793229 (12h37m33.970850079s) [upload=13013]: Uploading segments... Aug 7 22:17:12.227044345 (12h37m33.971101196s) [upload=13013]: Got next segment. Inline: false Aug 7 22:17:12.227107713 (12h37m33.971164563s) [upload=13013]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:17:12.227113591 (12h37m33.971170444s) [upload=13013]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:18:11.956251664 (12h38m33.700308534s) [upload=13013]: Next segment err: context deadline exceeded Aug 7 22:18:11.956360027 (12h38m33.700416877s) [upload=13013]: Done uploading segments... Aug 7 22:18:11.956379323 (12h38m33.700436185s) [upload=13013]: Upload finished. err: context deadline exceeded Duration: 1m0.000718865s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 22 out of 60 is failed (threshold=3) [XXXXXXXXXX..X.XX..................................X.XXXXXXXX] duration: 30.001520358s FAIL (2025-08-07 22:18:10 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001543257s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXX.....................................XXXXXXX] duration: 30.001346694s FAIL (2025-08-07 22:18:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001368463s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 24 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXX....................................XXXXXXXX] duration: 30.000838534s FAIL (2025-08-07 22:17:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== pload=12887]: Done starting upload Aug 7 22:09:20.958676819 (12h29m42.70273367s) [upload=12887]: Uploading segments... Aug 7 22:09:20.958736056 (12h29m42.702792918s) [upload=12887]: Got next segment. Inline: true Aug 7 22:09:20.958741890 (12h29m42.702798752s) [upload=12887]: Waiting for error group managing segments... Aug 7 22:09:20.958788210 (12h29m42.70284506s) [upload=12887]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:09:20.958793987 (12h29m42.702850848s) [upload=12887]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:09:20.958798029 (12h29m42.70285489s) [upload=12887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:09:21.118877309 (12h29m42.86293416s) [upload=12887]: Done uploading segments... Aug 7 22:09:21.118895252 (12h29m42.862952102s) [upload=12887]: Upload finished. err: <nil> Aug 7 22:10:21.179327195 (12h30m42.923384066s) [upload=12901]: Starting upload Aug 7 22:10:21.179521340 (12h30m42.92357821s) [upload=12901]: Done starting upload Aug 7 22:10:21.179644873 (12h30m42.923701721s) [upload=12901]: Uploading segments... Aug 7 22:10:21.179710572 (12h30m42.923767442s) [upload=12901]: Got next segment. Inline: true Aug 7 22:10:21.179717046 (12h30m42.923773918s) [upload=12901]: Waiting for error group managing segments... Aug 7 22:10:21.179768868 (12h30m42.923825739s) [upload=12901]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:10:21.179774459 (12h30m42.923831332s) [upload=12901]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:10:21.179778016 (12h30m42.923834886s) [upload=12901]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:10:21.324404712 (12h30m43.068461582s) [upload=12901]: Done uploading segments... Aug 7 22:10:21.324445701 (12h30m43.068502568s) [upload=12901]: Upload finished. err: <nil> Aug 7 22:11:27.454578904 (12h31m49.198635753s) [upload=12920]: Starting upload Aug 7 22:11:27.454695442 (12h31m49.19875231s) [upload=12920]: Done starting upload Aug 7 22:11:27.454814832 (12h31m49.198871679s) [upload=12920]: Uploading segments... Aug 7 22:11:27.454893840 (12h31m49.198950813s) [upload=12920]: Got next segment. Inline: true Aug 7 22:11:27.454915725 (12h31m49.198972587s) [upload=12920]: Waiting for error group managing segments... Aug 7 22:11:27.454970562 (12h31m49.199027411s) [upload=12920]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:11:27.454977263 (12h31m49.199034131s) [upload=12920]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:11:27.455074233 (12h31m49.199131094s) [upload=12920]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:11:57.179550579 (12h32m18.92360743s) [upload=12920]: Done uploading segments... Aug 7 22:11:57.179569786 (12h32m18.923626636s) [upload=12920]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:12:19.183043096 (12h32m40.927099946s) [upload=12933]: Starting upload Aug 7 22:12:19.183127401 (12h32m40.927184261s) [upload=12933]: Done starting upload Aug 7 22:12:19.183328044 (12h32m40.92738489s) [upload=12933]: Uploading segments... Aug 7 22:12:19.183400189 (12h32m40.92745706s) [upload=12933]: Got next segment. Inline: true Aug 7 22:12:19.183407329 (12h32m40.927464177s) [upload=12933]: Waiting for error group managing segments... Aug 7 22:12:19.183479311 (12h32m40.927536158s) [upload=12933]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:12:19.183484643 (12h32m40.927541506s) [upload=12933]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:12:19.183487379 (12h32m40.927544225s) [upload=12933]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:12:48.909423390 (12h33m10.653480257s) [upload=12933]: Done uploading segments... Aug 7 22:12:48.909444894 (12h33m10.653501768s) [upload=12933]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:13:10.831892676 (12h33m32.575949525s) [upload=12948]: Starting upload Aug 7 22:13:10.831993207 (12h33m32.576050068s) [upload=12948]: Done starting upload Aug 7 22:13:10.832071605 (12h33m32.576128457s) [upload=12948]: Uploading segments... Aug 7 22:13:10.832146431 (12h33m32.576203291s) [upload=12948]: Got next segment. Inline: true Aug 7 22:13:10.832152814 (12h33m32.576209674s) [upload=12948]: Waiting for error group managing segments... Aug 7 22:13:10.832208966 (12h33m32.576265827s) [upload=12948]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:13:10.832214384 (12h33m32.576271244s) [upload=12948]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:13:10.832217904 (12h33m32.576274764s) [upload=12948]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:13:40.559928781 (12h34m2.303985629s) [upload=12948]: Done uploading segments... Aug 7 22:13:40.559950271 (12h34m2.304007117s) [upload=12948]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:13:48.685772427 (12h34m10.429829287s) [upload=12958]: Starting upload Aug 7 22:13:48.685858893 (12h34m10.429915753s) [upload=12958]: Done starting upload Aug 7 22:13:48.685982193 (12h34m10.430039042s) [upload=12958]: Uploading segments... Aug 7 22:13:48.686047966 (12h34m10.430104835s) [upload=12958]: Got next segment. Inline: true Aug 7 22:13:48.686053563 (12h34m10.430110423s) [upload=12958]: Waiting for error group managing segments... Aug 7 22:13:48.686096300 (12h34m10.43015316s) [upload=12958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:13:48.686101341 (12h34m10.430158201s) [upload=12958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:13:48.686104243 (12h34m10.430161104s) [upload=12958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:14:18.399606594 (12h34m40.143663447s) [upload=12958]: Done uploading segments... Aug 7 22:14:18.399627822 (12h34m40.143684675s) [upload=12958]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:14:59.902619970 (12h35m21.646676816s) [upload=12977]: Starting upload Aug 7 22:14:59.902685895 (12h35m21.646742756s) [upload=12977]: Done starting upload Aug 7 22:14:59.902751694 (12h35m21.646808545s) [upload=12977]: Uploading segments... Aug 7 22:14:59.902795346 (12h35m21.646852214s) [upload=12977]: Got next segment. Inline: true Aug 7 22:14:59.902801495 (12h35m21.646858346s) [upload=12977]: Waiting for error group managing segments... Aug 7 22:14:59.902858709 (12h35m21.64691556s) [upload=12977]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:14:59.902863962 (12h35m21.646920811s) [upload=12977]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:14:59.902867458 (12h35m21.646924309s) [upload=12977]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:15:29.636302762 (12h35m51.380359612s) [upload=12977]: Done uploading segments... Aug 7 22:15:29.636325398 (12h35m51.380382249s) [upload=12977]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:45.428850713 (12h36m7.172907563s) [upload=12988]: Starting upload Aug 7 22:15:45.428978829 (12h36m7.173035696s) [upload=12988]: Done starting upload Aug 7 22:15:45.429103446 (12h36m7.173160298s) [upload=12988]: Uploading segments... Aug 7 22:15:45.429150754 (12h36m7.173207623s) [upload=12988]: Got next segment. Inline: true Aug 7 22:15:45.429156339 (12h36m7.173213199s) [upload=12988]: Waiting for error group managing segments... Aug 7 22:15:45.429230361 (12h36m7.173287221s) [upload=12988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:15:45.429247463 (12h36m7.173304325s) [upload=12988]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:15:45.429251133 (12h36m7.173307993s) [upload=12988]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:16:15.155440875 (12h36m36.899497725s) [upload=12988]: Done uploading segments... Aug 7 22:16:15.155471506 (12h36m36.899528367s) [upload=12988]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:16:15.598565945 (12h36m37.342622791s) [upload=12999]: Starting upload Aug 7 22:16:15.598645050 (12h36m37.342701913s) [upload=12999]: Done starting upload Aug 7 22:16:15.598718650 (12h36m37.342775497s) [upload=12999]: Uploading segments... Aug 7 22:16:15.598774264 (12h36m37.34283113s) [upload=12999]: Got next segment. Inline: true Aug 7 22:16:15.598779124 (12h36m37.342835984s) [upload=12999]: Waiting for error group managing segments... Aug 7 22:16:15.598830461 (12h36m37.342887324s) [upload=12999]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:16:15.598854102 (12h36m37.342910963s) [upload=12999]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:16:15.598857833 (12h36m37.342914694s) [upload=12999]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:16:45.328584439 (12h37m7.072641289s) [upload=12999]: Done uploading segments... Aug 7 22:16:45.328609005 (12h37m7.072665856s) [upload=12999]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:17:27.480756310 (12h37m49.22481316s) [upload=13015]: Starting upload Aug 7 22:17:27.480859326 (12h37m49.22491619s) [upload=13015]: Done starting upload Aug 7 22:17:27.480944450 (12h37m49.225001301s) [upload=13015]: Uploading segments... Aug 7 22:17:27.481000891 (12h37m49.225057757s) [upload=13015]: Got next segment. Inline: true Aug 7 22:17:27.481006468 (12h37m49.225063328s) [upload=13015]: Waiting for error group managing segments... Aug 7 22:17:27.481069727 (12h37m49.225126586s) [upload=13015]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:17:27.481076622 (12h37m49.225133484s) [upload=13015]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:27.481081288 (12h37m49.225138155s) [upload=13015]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:17:57.211870572 (12h38m18.955927423s) [upload=13015]: Done uploading segments... Aug 7 22:17:57.211892595 (12h38m18.955949444s) [upload=13015]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000861554s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................X...X........................] duration: 877.320738ms FAIL (2025-08-07 22:17:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 12661]: Done uploading segments... Aug 7 21:56:36.224982877 (12h16m57.969039745s) [upload=12661]: Upload finished. err: <nil> Aug 7 21:57:38.819941245 (12h18m0.563998096s) [upload=12678]: Starting upload Aug 7 21:57:38.820005339 (12h18m0.564062199s) [upload=12678]: Done starting upload Aug 7 21:57:38.820093525 (12h18m0.564150382s) [upload=12678]: Uploading segments... Aug 7 21:57:38.820164980 (12h18m0.564221849s) [upload=12678]: Got next segment. Inline: true Aug 7 21:57:38.820170360 (12h18m0.564227223s) [upload=12678]: Waiting for error group managing segments... Aug 7 21:57:38.820217982 (12h18m0.564274832s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:38.820223803 (12h18m0.564280662s) [upload=12678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:38.820227203 (12h18m0.564284063s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:39.040341313 (12h18m0.784398161s) [upload=12678]: Done uploading segments... Aug 7 21:57:39.040360634 (12h18m0.784417481s) [upload=12678]: Upload finished. err: <nil> Aug 7 21:57:54.880649336 (12h18m16.624706187s) [upload=12684]: Starting upload Aug 7 21:57:54.880693218 (12h18m16.624750088s) [upload=12684]: Done starting upload Aug 7 21:57:54.880791132 (12h18m16.624847993s) [upload=12684]: Uploading segments... Aug 7 21:57:54.880832914 (12h18m16.624889781s) [upload=12684]: Got next segment. Inline: true Aug 7 21:57:54.880838436 (12h18m16.624895296s) [upload=12684]: Waiting for error group managing segments... Aug 7 21:57:54.880894572 (12h18m16.624951432s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:54.880902144 (12h18m16.624959006s) [upload=12684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:54.880905596 (12h18m16.624962457s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:55.045748011 (12h18m16.789804872s) [upload=12684]: Done uploading segments... Aug 7 21:57:55.045767142 (12h18m16.789823992s) [upload=12684]: Upload finished. err: <nil> Aug 7 21:58:56.209662862 (12h19m17.953719708s) [upload=12702]: Starting upload Aug 7 21:58:56.209717729 (12h19m17.953774589s) [upload=12702]: Done starting upload Aug 7 21:58:56.209791551 (12h19m17.953848398s) [upload=12702]: Uploading segments... Aug 7 21:58:56.209845084 (12h19m17.953901954s) [upload=12702]: Got next segment. Inline: true Aug 7 21:58:56.209850002 (12h19m17.953906863s) [upload=12702]: Waiting for error group managing segments... Aug 7 21:58:56.209900221 (12h19m17.953957082s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:58:56.209905576 (12h19m17.953962423s) [upload=12702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:58:56.209940267 (12h19m17.953997127s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:58:56.605688962 (12h19m18.349745811s) [upload=12702]: Done uploading segments... Aug 7 21:58:56.605708466 (12h19m18.349765315s) [upload=12702]: Upload finished. err: <nil> Aug 7 21:59:15.802505476 (12h19m37.546562336s) [upload=12707]: Starting upload Aug 7 21:59:15.802568926 (12h19m37.546625778s) [upload=12707]: Done starting upload Aug 7 21:59:15.802684030 (12h19m37.546740881s) [upload=12707]: Uploading segments... Aug 7 21:59:15.802727059 (12h19m37.546783925s) [upload=12707]: Got next segment. Inline: true Aug 7 21:59:15.802732418 (12h19m37.546789279s) [upload=12707]: Waiting for error group managing segments... Aug 7 21:59:15.802792878 (12h19m37.546849738s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:15.802798596 (12h19m37.546855456s) [upload=12707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:15.802802334 (12h19m37.546859194s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:15.945145817 (12h19m37.689202664s) [upload=12707]: Done uploading segments... Aug 7 21:59:15.945175582 (12h19m37.689232429s) [upload=12707]: Upload finished. err: <nil> Aug 7 21:59:31.540495488 (12h19m53.284552336s) [upload=12712]: Starting upload Aug 7 21:59:31.540551951 (12h19m53.284608811s) [upload=12712]: Done starting upload Aug 7 21:59:31.540628497 (12h19m53.284685347s) [upload=12712]: Uploading segments... Aug 7 21:59:31.540671158 (12h19m53.284728027s) [upload=12712]: Got next segment. Inline: true Aug 7 21:59:31.540677279 (12h19m53.284734139s) [upload=12712]: Waiting for error group managing segments... Aug 7 21:59:31.540730297 (12h19m53.284787147s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:31.540735969 (12h19m53.284792829s) [upload=12712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:31.540739244 (12h19m53.284796105s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:31.703548635 (12h19m53.447605493s) [upload=12712]: Done uploading segments... Aug 7 21:59:31.703590031 (12h19m53.447646882s) [upload=12712]: Upload finished. err: <nil> Aug 7 22:00:29.248229344 (12h20m50.992286195s) [upload=12728]: Starting upload Aug 7 22:00:29.248281332 (12h20m50.992338188s) [upload=12728]: Done starting upload Aug 7 22:00:29.248374886 (12h20m50.992431738s) [upload=12728]: Uploading segments... Aug 7 22:00:29.248452896 (12h20m50.992509768s) [upload=12728]: Got next segment. Inline: true Aug 7 22:00:29.248460057 (12h20m50.992516919s) [upload=12728]: Waiting for error group managing segments... Aug 7 22:00:29.248538087 (12h20m50.992594934s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:29.248545868 (12h20m50.992602715s) [upload=12728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:29.248549647 (12h20m50.992606513s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:29.574503821 (12h20m51.318560681s) [upload=12728]: Done uploading segments... Aug 7 22:00:29.574527802 (12h20m51.318584666s) [upload=12728]: Upload finished. err: <nil> Aug 7 22:01:13.393836672 (12h21m35.137893523s) [upload=12740]: Starting upload Aug 7 22:01:13.393888179 (12h21m35.137945042s) [upload=12740]: Done starting upload Aug 7 22:01:13.394016213 (12h21m35.138073077s) [upload=12740]: Uploading segments... Aug 7 22:01:13.394161981 (12h21m35.13821896s) [upload=12740]: Got next segment. Inline: true Aug 7 22:01:13.394170576 (12h21m35.138227445s) [upload=12740]: Waiting for error group managing segments... Aug 7 22:01:13.394246764 (12h21m35.13830363s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:13.394254537 (12h21m35.138311402s) [upload=12740]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:13.394274292 (12h21m35.138331152s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:13.707894690 (12h21m35.451951555s) [upload=12740]: Done uploading segments... Aug 7 22:01:13.707926477 (12h21m35.451983343s) [upload=12740]: Upload finished. err: <nil> Aug 7 22:01:29.783138431 (12h21m51.527195291s) [upload=12746]: Starting upload Aug 7 22:01:29.783212111 (12h21m51.527268976s) [upload=12746]: Done starting upload Aug 7 22:01:29.783446600 (12h21m51.52750345s) [upload=12746]: Uploading segments... Aug 7 22:01:29.783527384 (12h21m51.527584254s) [upload=12746]: Got next segment. Inline: true Aug 7 22:01:29.783533979 (12h21m51.52759083s) [upload=12746]: Waiting for error group managing segments... Aug 7 22:01:29.783596800 (12h21m51.527653652s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:29.783605977 (12h21m51.527662827s) [upload=12746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:29.783609731 (12h21m51.52766658s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:29.966350494 (12h21m51.710407342s) [upload=12746]: Done uploading segments... Aug 7 22:01:29.966369237 (12h21m51.710426083s) [upload=12746]: Upload finished. err: <nil> Aug 7 22:01:45.372266942 (12h22m7.116323788s) [upload=12750]: Starting upload Aug 7 22:01:45.372314023 (12h22m7.116370883s) [upload=12750]: Done starting upload Aug 7 22:01:45.372374655 (12h22m7.116431503s) [upload=12750]: Uploading segments... Aug 7 22:01:45.372417267 (12h22m7.116474136s) [upload=12750]: Got next segment. Inline: true Aug 7 22:01:45.372422121 (12h22m7.116478981s) [upload=12750]: Waiting for error group managing segments... Aug 7 22:01:45.372461831 (12h22m7.116518692s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:45.372466235 (12h22m7.116523094s) [upload=12750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:45.372468880 (12h22m7.11652574s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:45.597843459 (12h22m7.341900345s) [upload=12750]: Done uploading segments... Aug 7 22:01:45.597877236 (12h22m7.341934095s) [upload=12750]: Upload finished. err: <nil> Aug 7 22:02:22.792150325 (12h22m44.53620717s) [upload=12764]: Starting upload Aug 7 22:02:22.792208195 (12h22m44.536265055s) [upload=12764]: Done starting upload Aug 7 22:02:22.792266980 (12h22m44.536323827s) [upload=12764]: Uploading segments... Aug 7 22:02:22.792307216 (12h22m44.536364085s) [upload=12764]: Got next segment. Inline: true Aug 7 22:02:22.792311926 (12h22m44.536368786s) [upload=12764]: Waiting for error group managing segments... Aug 7 22:02:22.792351265 (12h22m44.536408125s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.792356546 (12h22m44.536413408s) [upload=12764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.792359208 (12h22m44.536416068s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:23.139246613 (12h22m44.883303464s) [upload=12764]: Done uploading segments... Aug 7 22:02:23.139265320 (12h22m44.883322173s) [upload=12764]: Upload finished. err: <nil> Duration: 877.354043ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 135.79499ms remote: "tcp:185.244.226.2:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-07 22:17:50 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 135.809282ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [.X.X.X..................................................X...] duration: 284.173228ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:17:50 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 284.197662ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [XXXXXXXXXXXXX....X...................................XXXXXXX] duration: 30.000656183s FAIL (2025-08-07 22:17:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== e=true , address=77.90.31.15:4698 , piece_id=TGDU54ZJWBKTIAGDMZN4XMGW4XCPQZNMCBZ3X77UKL2AKGKPOTUQ, seg_pos={0 0}, upload=12907]: Done uploading piece. err: <nil> Aug 7 22:10:39.775515444 (12h31m1.51957229s) [noise=true , address=45.91.116.11:4119 , piece_id=VH3E5W6HVWOYXADUJ2Y7XROPLIO5FNM5AXTXO6ZCW5UMXASEARVQ, seg_pos={0 0}, upload=12907]: Done uploading piece. err: <nil> Aug 7 22:10:39.804475158 (12h31m1.548532025s) [noise=true , address=45.91.116.12:4225 , piece_id=Y7SFSDIQWKXFV6XYPOLMGV4PHGVGTA5QQ3NAXUMJ5I3HAHDYT3BA, seg_pos={0 0}, upload=12907]: Done uploading piece. err: <nil> Aug 7 22:10:39.943053768 (12h31m1.687110621s) [noise=true , address=104.36.82.3:4767 , piece_id=JNZSF4MCANQZGKIKBN44ZJ5HSP6GYS4JTBUL5LKEYWFZWROVX6FA, seg_pos={0 0}, upload=12907]: Done uploading piece. err: <nil> Aug 7 22:10:40.040942452 (12h31m1.784999305s) [noise=true , address=104.36.82.13:4187 , piece_id=SHMURFISNVVAQVXMURLXQB4YLPP7VRSERNNGG5TBH65MYIVIMRLA, seg_pos={0 0}, upload=12907]: Done uploading piece. err: <nil> Aug 7 22:10:40.080053386 (12h31m1.82411024s) [noise=true , address=77.90.31.19:4843 , piece_id=4KHRC5JUTN6BS7TNCJD4LQYRMB2DJ3FQTWXSND6MYTUBOC6QZ5AQ, seg_pos={0 0}, upload=12907]: Done uploading piece. err: <nil> Aug 7 22:10:40.080088161 (12h31m1.824145024s) [seg_pos={0 0}, upload=12907]: Segment reached optimal threshold of 13 pieces. Aug 7 22:10:40.080378206 (12h31m1.824435067s) [noise=true , address=104.36.82.8:4969 , piece_id=XCE4H7CFW223R6A4UWZ37SMRY44FKLKMMM5FEWA4PXWC6KPGTI2Q, seg_pos={0 0}, upload=12907]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: write tcp 139.59.144.22:57064->104.36.82.8:4969: use of closed network connection; write tcp 139.59.144.22:57064->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57064->104.36.82.8:4969: use of closed network connection Aug 7 22:10:40.080386263 (12h31m1.824443127s) [noise=true , address=45.91.116.6:4973 , piece_id=AC2G6YHJCUJDUW7BWFAYPGAPLHGSVUZMEOCMOCVCMER6AMSPVU7Q, seg_pos={0 0}, upload=12907]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 139.59.144.22:54360->45.91.116.6:4973: use of closed network connection; write tcp 139.59.144.22:54360->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54360->45.91.116.6:4973: use of closed network connection Aug 7 22:10:40.080444160 (12h31m1.824501022s) [noise=true , address=77.90.31.16:4733 , piece_id=BF2546OYO3A45NCBL6R7XUKAF4UXBQXGQCGWPHMOJQWZVIZTFTBQ, seg_pos={0 0}, upload=12907]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 139.59.144.22:60284->77.90.31.16:4733: use of closed network connection; write tcp 139.59.144.22:60284->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60284->77.90.31.16:4733: use of closed network connection Aug 7 22:10:40.080462019 (12h31m1.824518879s) [seg_pos={0 0}, upload=12907]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 22:10:40.080581669 (12h31m1.824638529s) [upload=12907]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:10:40.080586954 (12h31m1.824643814s) [upload=12907]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:10:40.368839911 (12h31m2.112896762s) [upload=12907]: Done uploading segments... Aug 7 22:10:40.368858450 (12h31m2.1129153s) [upload=12907]: Upload finished. err: <nil> Aug 7 22:11:49.338398173 (12h32m11.082455023s) [upload=12928]: Starting upload Aug 7 22:11:49.338461071 (12h32m11.082517935s) [upload=12928]: Done starting upload Aug 7 22:11:49.338536003 (12h32m11.082592853s) [upload=12928]: Uploading segments... Aug 7 22:11:49.338726540 (12h32m11.0827834s) [upload=12928]: Got next segment. Inline: false Aug 7 22:11:49.338761181 (12h32m11.082818049s) [upload=12928]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:11:49.338769861 (12h32m11.082826722s) [upload=12928]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:12:19.070280459 (12h32m40.814337331s) [upload=12928]: Next segment err: context deadline exceeded Aug 7 22:12:19.070407663 (12h32m40.814464525s) [upload=12928]: Done uploading segments... Aug 7 22:12:19.070433633 (12h32m40.81449048s) [upload=12928]: Upload finished. err: context deadline exceeded Aug 7 22:12:42.581064287 (12h33m4.325121134s) [upload=12941]: Starting upload Aug 7 22:12:42.581130100 (12h33m4.325186947s) [upload=12941]: Done starting upload Aug 7 22:12:42.581226419 (12h33m4.325283265s) [upload=12941]: Uploading segments... Aug 7 22:12:42.581487908 (12h33m4.32554477s) [upload=12941]: Got next segment. Inline: false Aug 7 22:12:42.581538275 (12h33m4.325595137s) [upload=12941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:12:42.581542942 (12h33m4.3255998s) [upload=12941]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:13:12.312332611 (12h33m34.056389476s) [upload=12941]: Next segment err: context deadline exceeded Aug 7 22:13:12.312420434 (12h33m34.056477284s) [upload=12941]: Done uploading segments... Aug 7 22:13:12.312445342 (12h33m34.056502207s) [upload=12941]: Upload finished. err: context deadline exceeded Aug 7 22:13:55.346563140 (12h34m17.090619992s) [upload=12961]: Starting upload Aug 7 22:13:55.346629229 (12h34m17.09068609s) [upload=12961]: Done starting upload Aug 7 22:13:55.346756038 (12h34m17.090812893s) [upload=12961]: Uploading segments... Aug 7 22:13:55.346883566 (12h34m17.090940419s) [upload=12961]: Got next segment. Inline: false Aug 7 22:13:55.346923294 (12h34m17.090980145s) [upload=12961]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:13:55.346928869 (12h34m17.090985729s) [upload=12961]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:14:25.076699300 (12h34m46.820756161s) [upload=12961]: Next segment err: context deadline exceeded Aug 7 22:14:25.076730315 (12h34m46.820787177s) [upload=12961]: Done uploading segments... Aug 7 22:14:25.076774228 (12h34m46.820831089s) [upload=12961]: Upload finished. err: context deadline exceeded Aug 7 22:14:25.500766908 (12h34m47.244823769s) [upload=12965]: Starting upload Aug 7 22:14:25.500843263 (12h34m47.244900124s) [upload=12965]: Done starting upload Aug 7 22:14:25.501285717 (12h34m47.245342581s) [upload=12965]: Uploading segments... Aug 7 22:14:25.501403791 (12h34m47.245460663s) [upload=12965]: Got next segment. Inline: false Aug 7 22:14:25.501454269 (12h34m47.245511121s) [upload=12965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:14:25.501477831 (12h34m47.245534692s) [upload=12965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:14:55.225463502 (12h35m16.969520369s) [upload=12965]: Next segment err: context deadline exceeded Aug 7 22:14:55.225785323 (12h35m16.969842175s) [upload=12965]: Done uploading segments... Aug 7 22:14:55.225811854 (12h35m16.969868704s) [upload=12965]: Upload finished. err: context deadline exceeded Aug 7 22:14:56.116847555 (12h35m17.8609044s) [upload=12975]: Starting upload Aug 7 22:14:56.116941603 (12h35m17.860998474s) [upload=12975]: Done starting upload Aug 7 22:14:56.117320062 (12h35m17.861376916s) [upload=12975]: Uploading segments... Aug 7 22:14:56.117414824 (12h35m17.861471693s) [upload=12975]: Got next segment. Inline: false Aug 7 22:14:56.117470124 (12h35m17.861526975s) [upload=12975]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:14:56.117475946 (12h35m17.861532796s) [upload=12975]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:15:25.850886466 (12h35m47.594943335s) [upload=12975]: Next segment err: context deadline exceeded Aug 7 22:15:25.851075699 (12h35m47.595132559s) [upload=12975]: Done uploading segments... Aug 7 22:15:25.851094465 (12h35m47.595151315s) [upload=12975]: Upload finished. err: context deadline exceeded Aug 7 22:16:00.222953691 (12h36m21.96701056s) [upload=12994]: Starting upload Aug 7 22:16:00.223061460 (12h36m21.967118333s) [upload=12994]: Done starting upload Aug 7 22:16:00.223394500 (12h36m21.967451353s) [upload=12994]: Uploading segments... Aug 7 22:16:00.223539867 (12h36m21.967596738s) [upload=12994]: Got next segment. Inline: false Aug 7 22:16:00.223639512 (12h36m21.967696376s) [upload=12994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:16:00.223647465 (12h36m21.967704329s) [upload=12994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:16:29.948699340 (12h36m51.69275621s) [upload=12994]: Next segment err: context deadline exceeded Aug 7 22:16:29.948780883 (12h36m51.692837732s) [upload=12994]: Done uploading segments... Aug 7 22:16:29.948801062 (12h36m51.692857912s) [upload=12994]: Upload finished. err: context deadline exceeded Aug 7 22:17:11.072117344 (12h37m32.816174211s) [upload=13012]: Starting upload Aug 7 22:17:11.072199666 (12h37m32.816256517s) [upload=13012]: Done starting upload Aug 7 22:17:11.072261694 (12h37m32.816318546s) [upload=13012]: Uploading segments... Aug 7 22:17:11.073454439 (12h37m32.817511291s) [upload=13012]: Got next segment. Inline: false Aug 7 22:17:11.073526596 (12h37m32.817583458s) [upload=13012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:17:11.073532686 (12h37m32.817589547s) [upload=13012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:17:40.800030583 (12h38m2.54408743s) [upload=13012]: Next segment err: context deadline exceeded Aug 7 22:17:40.800057653 (12h38m2.544114513s) [upload=13012]: Done uploading segments... Aug 7 22:17:40.800086865 (12h38m2.544143711s) [upload=13012]: Upload finished. err: context deadline exceeded Duration: 30.000681019s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 21 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXX......X................................X.XXXXX] duration: 1m0.000828795s FAIL (2025-08-07 22:17:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload Aug 7 22:08:25.754011963 (12h28m47.498068834s) [upload=12869]: Done starting upload Aug 7 22:08:25.754143380 (12h28m47.498200232s) [upload=12869]: Uploading segments... Aug 7 22:08:25.754224345 (12h28m47.498281211s) [upload=12869]: Got next segment. Inline: true Aug 7 22:08:25.754230381 (12h28m47.498287241s) [upload=12869]: Waiting for error group managing segments... Aug 7 22:08:25.754279562 (12h28m47.498336412s) [upload=12869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:08:25.754286362 (12h28m47.498343222s) [upload=12869]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:08:25.754289814 (12h28m47.498346677s) [upload=12869]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:08:25.900120121 (12h28m47.644176982s) [upload=12869]: Done uploading segments... Aug 7 22:08:25.900159641 (12h28m47.644216501s) [upload=12869]: Upload finished. err: <nil> Aug 7 22:09:00.184081014 (12h29m21.928137885s) [upload=12882]: Starting upload Aug 7 22:09:00.184229810 (12h29m21.928286679s) [upload=12882]: Done starting upload Aug 7 22:09:00.184332233 (12h29m21.928389079s) [upload=12882]: Uploading segments... Aug 7 22:09:00.184404155 (12h29m21.928461026s) [upload=12882]: Got next segment. Inline: true Aug 7 22:09:00.184410758 (12h29m21.928467627s) [upload=12882]: Waiting for error group managing segments... Aug 7 22:09:00.184480435 (12h29m21.928537304s) [upload=12882]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:09:00.184487151 (12h29m21.928544217s) [upload=12882]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:09:00.184491624 (12h29m21.928548491s) [upload=12882]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:09:00.583850980 (12h29m22.327907833s) [upload=12882]: Done uploading segments... Aug 7 22:09:00.583888682 (12h29m22.327945555s) [upload=12882]: Upload finished. err: <nil> Aug 7 22:09:40.429348924 (12h30m2.17340578s) [upload=12891]: Starting upload Aug 7 22:09:40.429449124 (12h30m2.173505984s) [upload=12891]: Done starting upload Aug 7 22:09:40.429576700 (12h30m2.173633561s) [upload=12891]: Uploading segments... Aug 7 22:09:40.429631265 (12h30m2.173688133s) [upload=12891]: Got next segment. Inline: true Aug 7 22:09:40.429637613 (12h30m2.173694464s) [upload=12891]: Waiting for error group managing segments... Aug 7 22:09:40.429689508 (12h30m2.173746362s) [upload=12891]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:09:40.429698290 (12h30m2.17375514s) [upload=12891]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:09:40.429702162 (12h30m2.173759014s) [upload=12891]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:09:40.614648207 (12h30m2.358705075s) [upload=12891]: Done uploading segments... Aug 7 22:09:40.614672259 (12h30m2.358729124s) [upload=12891]: Upload finished. err: <nil> Aug 7 22:10:28.314751006 (12h30m50.058807867s) [upload=12903]: Starting upload Aug 7 22:10:28.314813386 (12h30m50.058870247s) [upload=12903]: Done starting upload Aug 7 22:10:28.314903941 (12h30m50.058960802s) [upload=12903]: Uploading segments... Aug 7 22:10:28.314992884 (12h30m50.059049757s) [upload=12903]: Got next segment. Inline: true Aug 7 22:10:28.315015785 (12h30m50.059072647s) [upload=12903]: Waiting for error group managing segments... Aug 7 22:10:28.315072549 (12h30m50.059129415s) [upload=12903]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:10:28.315079820 (12h30m50.059136685s) [upload=12903]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:10:28.315099592 (12h30m50.059156457s) [upload=12903]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:11:28.039892681 (12h31m49.783949531s) [upload=12903]: Done uploading segments... Aug 7 22:11:28.039913432 (12h31m49.783970283s) [upload=12903]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:11:35.297516727 (12h31m57.041573588s) [upload=12923]: Starting upload Aug 7 22:11:35.297601596 (12h31m57.041658469s) [upload=12923]: Done starting upload Aug 7 22:11:35.297686234 (12h31m57.041743086s) [upload=12923]: Uploading segments... Aug 7 22:11:35.297779680 (12h31m57.041836543s) [upload=12923]: Got next segment. Inline: true Aug 7 22:11:35.297790270 (12h31m57.041847121s) [upload=12923]: Waiting for error group managing segments... Aug 7 22:11:35.297854321 (12h31m57.041911182s) [upload=12923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:11:35.297860387 (12h31m57.041917249s) [upload=12923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:11:35.297863885 (12h31m57.041920745s) [upload=12923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:11:44.558966532 (12h32m6.303023384s) [upload=12923]: Done uploading segments... Aug 7 22:11:44.558985310 (12h32m6.303042163s) [upload=12923]: Upload finished. err: <nil> Aug 7 22:12:36.559571434 (12h32m58.30362831s) [upload=12938]: Starting upload Aug 7 22:12:36.559699331 (12h32m58.3037562s) [upload=12938]: Done starting upload Aug 7 22:12:36.559803583 (12h32m58.30386044s) [upload=12938]: Uploading segments... Aug 7 22:12:36.560034082 (12h32m58.304090945s) [upload=12938]: Got next segment. Inline: true Aug 7 22:12:36.560041392 (12h32m58.304098252s) [upload=12938]: Waiting for error group managing segments... Aug 7 22:12:36.560151251 (12h32m58.304208113s) [upload=12938]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:12:36.560159993 (12h32m58.304216852s) [upload=12938]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:12:36.560164091 (12h32m58.304220952s) [upload=12938]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:13:36.275827133 (12h33m58.019883992s) [upload=12938]: Done uploading segments... Aug 7 22:13:36.275867976 (12h33m58.019924837s) [upload=12938]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:13:36.703916041 (12h33m58.447972892s) [upload=12954]: Starting upload Aug 7 22:13:36.704016678 (12h33m58.448073543s) [upload=12954]: Done starting upload Aug 7 22:13:36.704126028 (12h33m58.448182877s) [upload=12954]: Uploading segments... Aug 7 22:13:36.704202003 (12h33m58.448258875s) [upload=12954]: Got next segment. Inline: true Aug 7 22:13:36.704208724 (12h33m58.448265583s) [upload=12954]: Waiting for error group managing segments... Aug 7 22:13:36.704253077 (12h33m58.44830994s) [upload=12954]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:13:36.704259505 (12h33m58.448316353s) [upload=12954]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:13:36.704263329 (12h33m58.448320184s) [upload=12954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:14:36.426520849 (12h34m58.170577716s) [upload=12954]: Done uploading segments... Aug 7 22:14:36.426547324 (12h34m58.170604188s) [upload=12954]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:14:36.851362491 (12h34m58.595419352s) [upload=12969]: Starting upload Aug 7 22:14:36.851438058 (12h34m58.595494919s) [upload=12969]: Done starting upload Aug 7 22:14:36.851531336 (12h34m58.595588182s) [upload=12969]: Uploading segments... Aug 7 22:14:36.851620028 (12h34m58.595676898s) [upload=12969]: Got next segment. Inline: true Aug 7 22:14:36.851627191 (12h34m58.595684056s) [upload=12969]: Waiting for error group managing segments... Aug 7 22:14:36.851698724 (12h34m58.595755585s) [upload=12969]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:14:36.851706200 (12h34m58.595763081s) [upload=12969]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:14:36.851710849 (12h34m58.595767717s) [upload=12969]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:15:36.582802839 (12h35m58.326859693s) [upload=12969]: Done uploading segments... Aug 7 22:15:36.582825807 (12h35m58.326882658s) [upload=12969]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:37.013505363 (12h35m58.757562231s) [upload=12985]: Starting upload Aug 7 22:15:37.013605304 (12h35m58.757662168s) [upload=12985]: Done starting upload Aug 7 22:15:37.013715315 (12h35m58.757772164s) [upload=12985]: Uploading segments... Aug 7 22:15:37.013761212 (12h35m58.757818081s) [upload=12985]: Got next segment. Inline: true Aug 7 22:15:37.013776983 (12h35m58.757833843s) [upload=12985]: Waiting for error group managing segments... Aug 7 22:15:37.013820691 (12h35m58.757877552s) [upload=12985]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:15:37.013826603 (12h35m58.757883467s) [upload=12985]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:15:37.013830513 (12h35m58.757887374s) [upload=12985]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:16:36.733288222 (12h36m58.477345085s) [upload=12985]: Done uploading segments... Aug 7 22:16:36.733314893 (12h36m58.477371754s) [upload=12985]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:16:37.187949023 (12h36m58.932005874s) [upload=13003]: Starting upload Aug 7 22:16:37.188024595 (12h36m58.932081456s) [upload=13003]: Done starting upload Aug 7 22:16:37.188115827 (12h36m58.932172677s) [upload=13003]: Uploading segments... Aug 7 22:16:37.188173315 (12h36m58.932230182s) [upload=13003]: Got next segment. Inline: true Aug 7 22:16:37.188181498 (12h36m58.932238365s) [upload=13003]: Waiting for error group managing segments... Aug 7 22:16:37.188252566 (12h36m58.932309427s) [upload=13003]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:16:37.188260045 (12h36m58.932316913s) [upload=13003]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:16:37.188263607 (12h36m58.932320469s) [upload=13003]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:17:36.907306861 (12h37m58.651363739s) [upload=13003]: Done uploading segments... Aug 7 22:17:36.907340408 (12h37m58.651397275s) [upload=13003]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.000858602s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [........XX.X................................................] duration: 51.602232143s remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:17:20 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 51.602256805s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 26 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXX......X.X...........................XXXXXX] duration: 1m0.001379807s FAIL (2025-08-07 22:17:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001401054s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 17 out of 60 is failed (threshold=3) [XXXXXXXXXXX.X..........................................XXXXX] duration: 30.001234802s FAIL (2025-08-07 22:17:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001271629s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [X.X..........X.....X.........X..X...........X...............] duration: 3.875544826s FAIL (2025-08-07 22:17:12 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 5.727759364s) [upload=12820]: Starting upload Aug 7 22:05:23.983754833 (12h25m45.727811679s) [upload=12820]: Done starting upload Aug 7 22:05:23.983988533 (12h25m45.728045379s) [upload=12820]: Uploading segments... Aug 7 22:05:23.984022348 (12h25m45.728079214s) [upload=12820]: Got next segment. Inline: false Aug 7 22:05:23.984055700 (12h25m45.728112548s) [upload=12820]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:23.984060538 (12h25m45.728117398s) [upload=12820]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:05:24.185392691 (12h25m45.929449561s) [seg_pos={0 0}, upload=12820]: Begin upload segment... Aug 7 22:05:24.185557956 (12h25m45.929614817s) [seg_pos={0 0}, upload=12820]: Done begin upload segment. Aug 7 22:05:24.185603301 (12h25m45.929660177s) [noise=true , address=64.49.13.169:4705 , piece_id=6C45BITH3QWBJ22PLPILYA6LOUQ2JF2D2LOAL2BVJQQWXSXQ37IQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186037040 (12h25m45.930093906s) [noise=true , address=104.36.82.12:4174 , piece_id=DCAYU4CKMHLHOHAN7MKLV5KJTRCCC2IOWSPFPWEMAU3FSIVIXRQA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186570414 (12h25m45.930627287s) [noise=true , address=104.36.82.16:4314 , piece_id=KX4ZY5LSNB6YBYAS4NKW2QQXXQZV42VGR2TRRKZIL6ZIO3M7BAQQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.186824559 (12h25m45.930881425s) [noise=true , address=104.36.82.46:4496 , piece_id=XNRPMJSQ3BNIILAQDMRYLFM623WHRZ63ND4RDTHJYLSWWOPNJGHA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187208122 (12h25m45.931264991s) [noise=true , address=104.36.82.43:4868 , piece_id=QSDRGPBV3KWFHHKWJWKLJ3N5APRHOI66TTWAWNOJDCRXQU2P7PAA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187658761 (12h25m45.931715627s) [noise=true , address=104.36.82.60:4346 , piece_id=CJPTIG7EI264SA2JCH5326GTL33YW6ASGJPXO5ZRWRDD3L5BQTYA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.187940002 (12h25m45.931996872s) [noise=true , address=64.49.13.140:4515 , piece_id=WQSSOLYXPIBGHNVEG5IWDPLM45QDWPATVU73OJJTRU7NJRQCHDKA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188116203 (12h25m45.932173069s) [noise=true , address=45.91.116.6:4982 , piece_id=LAWUMX2HZNNHRQ3FUODV65COBDIMOZN2HBCSPD42P5DEC4N4VABQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188531115 (12h25m45.932587976s) [noise=true , address=104.36.82.58:4292 , piece_id=RJJS3MVCC3B3RTAXEM3XLHRSWZEUKIJUSILSKJKT5LJCWNEHX5GA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188637273 (12h25m45.932694145s) [noise=true , address=104.36.82.5:4862 , piece_id=W6AHEAV5E5RQR3YQ3LFOO33URBFGYOVTT23S5JKPP7MDNIRFHV5Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.188961006 (12h25m45.933017872s) [noise=true , address=104.36.82.22:4530 , piece_id=U33JUCE4QFPT54RRVY57EF64QS7C4TUO7MWOOGZFAIHBZQUZFB3A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189183712 (12h25m45.933240578s) [noise=true , address=64.49.13.26:4945 , piece_id=BODENY6IN5IS2I2RJ3J3EV3XYKZD4B4RR4PQNKSEXYXVUVYPMC6A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189352560 (12h25m45.933409422s) [noise=true , address=104.36.82.56:4233 , piece_id=ING27ZVSXOL6ASCWIVGSF2KESQ4N6O4AXGQ3HF5EZK5TZ5T3SNNQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189640677 (12h25m45.93369755s) [noise=true , address=104.36.82.20:4461 , piece_id=XGB6IA7NPLFF7XKYATSOG54O7U36IX6BVHVWGA53T7MNVBFM5R2A, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.189855650 (12h25m45.933912517s) [noise=true , address=104.36.82.62:4439 , piece_id=TQYPKEGJFIZ7D26GWFTFSU4PU75J5DX24QKUEEUUX66EINYJEMIQ, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190187162 (12h25m45.934244033s) [noise=true , address=104.36.82.13:4201 , piece_id=PD7C77UABKPFT2WM7BOTNOGQV22DUPGWWOGZ5KUUMUVMWSSSHTJA, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190334028 (12h25m45.934390898s) [noise=true , address=104.36.82.47:4534 , piece_id=JS7S2OPLDOU5C52ZACTR77R2OVC56PBN6BDNGAPRBNN3UHXQPX3Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.190661949 (12h25m45.934718816s) [noise=true , address=77.90.31.16:4742 , piece_id=5USKIHBGUAV6G3BX5BHC6LOEEHDNAJUFK45RWH6OC25QGOMH7V4Q, seg_pos={0 0}, upload=12820]: Uploading piece... Aug 7 22:05:24.754845434 (12h25m46.498902284s) [upload=12820]: Next returned nil segment Aug 7 22:05:24.754862619 (12h25m46.49891948s) [upload=12820]: Waiting for error group managing segments... Aug 7 22:05:24.937656798 (12h25m46.681713649s) [noise=true , address=45.91.116.6:4982 , piece_id=LAWUMX2HZNNHRQ3FUODV65COBDIMOZN2HBCSPD42P5DEC4N4VABQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:24.980443483 (12h25m46.724500333s) [noise=true , address=104.36.82.12:4174 , piece_id=DCAYU4CKMHLHOHAN7MKLV5KJTRCCC2IOWSPFPWEMAU3FSIVIXRQA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:24.995491860 (12h25m46.739551452s) [noise=true , address=77.90.31.16:4742 , piece_id=5USKIHBGUAV6G3BX5BHC6LOEEHDNAJUFK45RWH6OC25QGOMH7V4Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.083772095 (12h25m46.827828964s) [noise=true , address=104.36.82.46:4496 , piece_id=XNRPMJSQ3BNIILAQDMRYLFM623WHRZ63ND4RDTHJYLSWWOPNJGHA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.147179204 (12h25m46.891236066s) [noise=true , address=104.36.82.13:4201 , piece_id=PD7C77UABKPFT2WM7BOTNOGQV22DUPGWWOGZ5KUUMUVMWSSSHTJA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.274393636 (12h25m47.018450492s) [noise=true , address=104.36.82.62:4439 , piece_id=TQYPKEGJFIZ7D26GWFTFSU4PU75J5DX24QKUEEUUX66EINYJEMIQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.452784464 (12h25m47.196841314s) [noise=true , address=64.49.13.169:4705 , piece_id=6C45BITH3QWBJ22PLPILYA6LOUQ2JF2D2LOAL2BVJQQWXSXQ37IQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.474459740 (12h25m47.218516586s) [noise=true , address=64.49.13.26:4945 , piece_id=BODENY6IN5IS2I2RJ3J3EV3XYKZD4B4RR4PQNKSEXYXVUVYPMC6A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.481247102 (12h25m47.225303949s) [noise=true , address=64.49.13.140:4515 , piece_id=WQSSOLYXPIBGHNVEG5IWDPLM45QDWPATVU73OJJTRU7NJRQCHDKA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.666066275 (12h25m47.410123138s) [noise=true , address=104.36.82.58:4292 , piece_id=RJJS3MVCC3B3RTAXEM3XLHRSWZEUKIJUSILSKJKT5LJCWNEHX5GA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.666429351 (12h25m47.410486202s) [noise=true , address=104.36.82.56:4233 , piece_id=ING27ZVSXOL6ASCWIVGSF2KESQ4N6O4AXGQ3HF5EZK5TZ5T3SNNQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.782101005 (12h25m47.526157853s) [noise=true , address=104.36.82.16:4314 , piece_id=KX4ZY5LSNB6YBYAS4NKW2QQXXQZV42VGR2TRRKZIL6ZIO3M7BAQQ, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.863469768 (12h25m47.607526622s) [noise=true , address=104.36.82.43:4868 , piece_id=QSDRGPBV3KWFHHKWJWKLJ3N5APRHOI66TTWAWNOJDCRXQU2P7PAA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.865162168 (12h25m47.609219032s) [noise=true , address=104.36.82.60:4346 , piece_id=CJPTIG7EI264SA2JCH5326GTL33YW6ASGJPXO5ZRWRDD3L5BQTYA, seg_pos={0 0}, upload=12820]: Done uploading piece. err: <nil> Aug 7 22:05:25.865201343 (12h25m47.609258205s) [seg_pos={0 0}, upload=12820]: Segment reached optimal threshold of 14 pieces. Aug 7 22:05:25.865362755 (12h25m47.609419601s) [noise=true , address=104.36.82.47:4534 , piece_id=JS7S2OPLDOU5C52ZACTR77R2OVC56PBN6BDNGAPRBNN3UHXQPX3Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VDrNGcCnaKEmG5NN4nex1JoZAgdCEfX5CdsKbxytfLfKzPbr): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865412734 (12h25m47.609469599s) [noise=true , address=104.36.82.20:4461 , piece_id=XGB6IA7NPLFF7XKYATSOG54O7U36IX6BVHVWGA53T7MNVBFM5R2A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nhwT2THUu6ptumqU9Bz1Nd34FHLewx9vWjGFC34j3Lz3RWfkw): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865468394 (12h25m47.609525241s) [noise=true , address=104.36.82.5:4862 , piece_id=W6AHEAV5E5RQR3YQ3LFOO33URBFGYOVTT23S5JKPP7MDNIRFHV5Q, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: expected piece hash; context canceled; EOF Aug 7 22:05:25.865629517 (12h25m47.609686385s) [noise=true , address=104.36.82.22:4530 , piece_id=U33JUCE4QFPT54RRVY57EF64QS7C4TUO7MWOOGZFAIHBZQUZFB3A, seg_pos={0 0}, upload=12820]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1akHUSoQFafzkGTJ385iYaowc87pJBDNfU5hodSCUDFa5ojN3v): protocol: write tcp 139.59.144.22:45386->104.36.82.22:4530: use of closed network connection; context canceled Aug 7 22:05:25.865657252 (12h25m47.609714121s) [seg_pos={0 0}, upload=12820]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:05:25.865814825 (12h25m47.609871688s) [upload=12820]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:05:25.865822925 (12h25m47.609879795s) [upload=12820]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:26.138690358 (12h25m47.882747231s) [upload=12820]: Done uploading segments... Aug 7 22:05:26.138713087 (12h25m47.882769948s) [upload=12820]: Upload finished. err: <nil> Duration: 3.875576324s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [.......................XXXX..............................XXX] duration: 1m0.001432732s FAIL (2025-08-07 22:17:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 7 22:11:56.394423773 (12h32m18.138480634s) [noise=true , address=104.36.82.6:4895 , piece_id=HUPRXOSI2CS2UDOF47BME2BYCCLXANUCJAYVZV4JNRMPQQUDZQFA, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.394711279 (12h32m18.138768152s) [noise=true , address=104.36.82.39:4733 , piece_id=VZPW75CLRYOWCVQGWBWAOGWKFCXYLYOM6OIDMXRBR5IRGZ7DGLLQ, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.394721898 (12h32m18.13877876s) [noise=true , address=104.36.82.5:4855 , piece_id=OEDGYJGQIIJHLEBR5OPPQCO75KXIGPFZBDCCTBYSUV7XQADJRSAA, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.395063526 (12h32m18.139120389s) [noise=true , address=104.36.82.62:4439 , piece_id=2P7NHL5WLDREVBGDS4IMRS6D4DZBQHRYFM7USCDK2KU74SKDLW4A, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.395086234 (12h32m18.139143094s) [noise=true , address=45.91.116.13:4166 , piece_id=HSFXWLME6GIFAWHD73GW4H6CBPBJZ4MPTB7JD7XY2YBTZESICNRA, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.395528042 (12h32m18.139584909s) [noise=true , address=104.36.82.60:4343 , piece_id=A5ZQY7FYEN7BVIGGB6T43UTFNOC6NGPBM3XJ23QSLVRFFVSVYJCQ, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.395636576 (12h32m18.139693442s) [noise=true , address=104.36.82.13:4188 , piece_id=C24Q324TD3OIGJSBOKVUN2VH5ORNKINWDHBTA3O56VSRPJN52SGA, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.395864366 (12h32m18.139921236s) [noise=true , address=104.36.82.47:4520 , piece_id=DLEDG3JMSGGGFQFGNJWV4XIDPXITG54XCYTRXXJIRR4DOXWIDAVA, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.396030181 (12h32m18.140087045s) [noise=true , address=104.36.82.36:4645 , piece_id=P63BF2T4FLUK7DMM4GQY277QTDGZXT3CD4Z5GFDMUQTTJN77TZIQ, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.396205503 (12h32m18.140262364s) [noise=true , address=45.91.116.9:4272 , piece_id=3QGHJZTVECLQTJGC46GZ5276LVTHR36LBTK5Q5FSRGD7QXOUJJ6A, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.397743950 (12h32m18.141800804s) [noise=true , address=64.49.13.139:4471 , piece_id=QTORROJDLNZQTDC2VFU3Q5DW47TA7VM7FZAVKTACLLCPMHGOC52A, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.398091992 (12h32m18.142148846s) [noise=true , address=104.36.82.43:4837 , piece_id=WALP4A6IL6VR6577XHAIATXZGBECNO6BL5Y5Z4I4DMPCNIK7X5QQ, seg_pos={0 0}, upload=12925]: Uploading piece... Aug 7 22:11:56.979185930 (12h32m18.723242776s) [upload=12925]: Next returned nil segment Aug 7 22:11:56.979206213 (12h32m18.723263074s) [upload=12925]: Waiting for error group managing segments... Aug 7 22:11:57.134173791 (12h32m18.878230638s) [noise=true , address=104.36.82.13:4188 , piece_id=C24Q324TD3OIGJSBOKVUN2VH5ORNKINWDHBTA3O56VSRPJN52SGA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.151552906 (12h32m18.895609777s) [noise=true , address=104.36.82.6:4895 , piece_id=HUPRXOSI2CS2UDOF47BME2BYCCLXANUCJAYVZV4JNRMPQQUDZQFA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.151985548 (12h32m18.896042414s) [noise=true , address=104.36.82.60:4343 , piece_id=A5ZQY7FYEN7BVIGGB6T43UTFNOC6NGPBM3XJ23QSLVRFFVSVYJCQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.177078648 (12h32m18.921135515s) [noise=true , address=104.36.82.34:4566 , piece_id=GR2IDGJL2TB5VYWZOIW2C652I5B5XUUXXI4SIELMP3V4GMO2ZH4Q, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.180968885 (12h32m18.925025749s) [noise=true , address=104.36.82.45:4940 , piece_id=E2BSUMWRGNFOCNBBT32GAMW3JN4PFKNKSYXK5AEUGQQKRK4I6FKA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.182573730 (12h32m18.926630601s) [noise=true , address=104.36.82.43:4837 , piece_id=WALP4A6IL6VR6577XHAIATXZGBECNO6BL5Y5Z4I4DMPCNIK7X5QQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.185217873 (12h32m18.929274727s) [noise=true , address=104.36.82.60:4346 , piece_id=OCI6XALWGEOAF64WRNURL5VVPFWYBT24GXK4GNQMUX2MJYZACROQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.191081154 (12h32m18.935138022s) [noise=true , address=104.36.82.23:4448 , piece_id=RD4YTZPX2A4SRE2XWYS45DWZRG6CUS5VLCYBMJPBJ24LGJTGITFQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.205956300 (12h32m18.950013175s) [noise=true , address=104.36.82.36:4645 , piece_id=P63BF2T4FLUK7DMM4GQY277QTDGZXT3CD4Z5GFDMUQTTJN77TZIQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.214668342 (12h32m18.958725212s) [noise=true , address=104.36.82.39:4733 , piece_id=VZPW75CLRYOWCVQGWBWAOGWKFCXYLYOM6OIDMXRBR5IRGZ7DGLLQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.245950582 (12h32m18.990007453s) [noise=true , address=104.36.82.16:4310 , piece_id=UAVZCLFOGCC6T4HX676IX6OWFNRZRTXL3PITG5IZFR2JATIZS2DQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.280359350 (12h32m19.024416213s) [noise=true , address=45.91.116.9:4272 , piece_id=3QGHJZTVECLQTJGC46GZ5276LVTHR36LBTK5Q5FSRGD7QXOUJJ6A, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.338710901 (12h32m19.082767752s) [noise=true , address=45.91.116.13:4166 , piece_id=HSFXWLME6GIFAWHD73GW4H6CBPBJZ4MPTB7JD7XY2YBTZESICNRA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.617684582 (12h32m19.361741428s) [noise=true , address=104.36.82.62:4439 , piece_id=2P7NHL5WLDREVBGDS4IMRS6D4DZBQHRYFM7USCDK2KU74SKDLW4A, seg_pos={0 0}, upload=12925]: Done uploading piece. err: <nil> Aug 7 22:11:57.617723267 (12h32m19.361780128s) [seg_pos={0 0}, upload=12925]: Segment reached optimal threshold of 14 pieces. Aug 7 22:11:57.617992734 (12h32m19.36204958s) [noise=true , address=104.36.82.47:4520 , piece_id=DLEDG3JMSGGGFQFGNJWV4XIDPXITG54XCYTRXXJIRR4DOXWIDAVA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 22:11:57.618089635 (12h32m19.362146495s) [noise=true , address=64.49.13.137:4498 , piece_id=7SLUBM4U2CKK25FCFPHVKBA4D4TKCRZXKASC7LR2ASBM7C7MDQKQ, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: expected piece hash; context canceled; EOF Aug 7 22:11:57.618135706 (12h32m19.362192556s) [noise=true , address=104.36.82.5:4855 , piece_id=OEDGYJGQIIJHLEBR5OPPQCO75KXIGPFZBDCCTBYSUV7XQADJRSAA, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh): protocol: write tcp 139.59.144.22:57090->104.36.82.5:4855: use of closed network connection; write tcp 139.59.144.22:57090->104.36.82.5:4855: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57090->104.36.82.5:4855: use of closed network connection Aug 7 22:11:57.618165938 (12h32m19.362222785s) [noise=true , address=64.49.13.139:4471 , piece_id=QTORROJDLNZQTDC2VFU3Q5DW47TA7VM7FZAVKTACLLCPMHGOC52A, seg_pos={0 0}, upload=12925]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129YkXH84aYSubinAC57swW24qCCEtvJTSMDGoxP9WdgpGcZxLr): protocol: expected piece hash; context canceled; EOF Aug 7 22:11:57.618181168 (12h32m19.362238028s) [seg_pos={0 0}, upload=12925]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:11:57.618262981 (12h32m19.362319833s) [upload=12925]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:11:57.618268565 (12h32m19.362325425s) [upload=12925]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:15:11.462898274 (12h35m33.206955137s) [upload=12925]: Done uploading segments... Aug 7 22:15:11.462927061 (12h35m33.206983911s) [upload=12925]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:15:11.893865947 (12h35m33.637922798s) [upload=12978]: Starting upload Aug 7 22:15:11.894006161 (12h35m33.638063025s) [upload=12978]: Done starting upload Aug 7 22:15:11.894128456 (12h35m33.63818531s) [upload=12978]: Uploading segments... Aug 7 22:15:11.894439225 (12h35m33.638496094s) [upload=12978]: Got next segment. Inline: false Aug 7 22:15:11.894516777 (12h35m33.638573639s) [upload=12978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:15:11.894545833 (12h35m33.638602694s) [upload=12978]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:16:11.626077800 (12h36m33.370134666s) [upload=12978]: Next segment err: context deadline exceeded Aug 7 22:16:11.626165441 (12h36m33.370222301s) [upload=12978]: Done uploading segments... Aug 7 22:16:11.626183066 (12h36m33.370239913s) [upload=12978]: Upload finished. err: context deadline exceeded Aug 7 22:16:12.058909015 (12h36m33.802965875s) [upload=12997]: Starting upload Aug 7 22:16:12.058982372 (12h36m33.803039232s) [upload=12997]: Done starting upload Aug 7 22:16:12.059120653 (12h36m33.803177509s) [upload=12997]: Uploading segments... Aug 7 22:16:12.059593126 (12h36m33.803649985s) [upload=12997]: Got next segment. Inline: false Aug 7 22:16:12.059677172 (12h36m33.803734023s) [upload=12997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:16:12.059684424 (12h36m33.803741287s) [upload=12997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:17:11.787443939 (12h37m33.531500809s) [upload=12997]: Next segment err: context deadline exceeded Aug 7 22:17:11.787498695 (12h37m33.531555556s) [upload=12997]: Done uploading segments... Aug 7 22:17:11.787527052 (12h37m33.531583904s) [upload=12997]: Upload finished. err: context deadline exceeded Duration: 1m0.001450548s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [..X.X.X..................................................X..] duration: 111.678651ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:17:08 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 111.707716ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXX.....................................XXXXXX] duration: 30.000865347s FAIL (2025-08-07 22:17:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000890267s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 22 out of 60 is failed (threshold=3) [XXXXXXXXXXX..X.XX..................................X.XXXXXXX] duration: 30.00102287s FAIL (2025-08-07 22:17:05 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001060917s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................X...X.......................] duration: 1.05660298s FAIL (2025-08-07 22:16:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 12661]: Done uploading segments... Aug 7 21:56:36.224982877 (12h16m57.969039745s) [upload=12661]: Upload finished. err: <nil> Aug 7 21:57:38.819941245 (12h18m0.563998096s) [upload=12678]: Starting upload Aug 7 21:57:38.820005339 (12h18m0.564062199s) [upload=12678]: Done starting upload Aug 7 21:57:38.820093525 (12h18m0.564150382s) [upload=12678]: Uploading segments... Aug 7 21:57:38.820164980 (12h18m0.564221849s) [upload=12678]: Got next segment. Inline: true Aug 7 21:57:38.820170360 (12h18m0.564227223s) [upload=12678]: Waiting for error group managing segments... Aug 7 21:57:38.820217982 (12h18m0.564274832s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:38.820223803 (12h18m0.564280662s) [upload=12678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:38.820227203 (12h18m0.564284063s) [upload=12678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:39.040341313 (12h18m0.784398161s) [upload=12678]: Done uploading segments... Aug 7 21:57:39.040360634 (12h18m0.784417481s) [upload=12678]: Upload finished. err: <nil> Aug 7 21:57:54.880649336 (12h18m16.624706187s) [upload=12684]: Starting upload Aug 7 21:57:54.880693218 (12h18m16.624750088s) [upload=12684]: Done starting upload Aug 7 21:57:54.880791132 (12h18m16.624847993s) [upload=12684]: Uploading segments... Aug 7 21:57:54.880832914 (12h18m16.624889781s) [upload=12684]: Got next segment. Inline: true Aug 7 21:57:54.880838436 (12h18m16.624895296s) [upload=12684]: Waiting for error group managing segments... Aug 7 21:57:54.880894572 (12h18m16.624951432s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:57:54.880902144 (12h18m16.624959006s) [upload=12684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:57:54.880905596 (12h18m16.624962457s) [upload=12684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:57:55.045748011 (12h18m16.789804872s) [upload=12684]: Done uploading segments... Aug 7 21:57:55.045767142 (12h18m16.789823992s) [upload=12684]: Upload finished. err: <nil> Aug 7 21:58:56.209662862 (12h19m17.953719708s) [upload=12702]: Starting upload Aug 7 21:58:56.209717729 (12h19m17.953774589s) [upload=12702]: Done starting upload Aug 7 21:58:56.209791551 (12h19m17.953848398s) [upload=12702]: Uploading segments... Aug 7 21:58:56.209845084 (12h19m17.953901954s) [upload=12702]: Got next segment. Inline: true Aug 7 21:58:56.209850002 (12h19m17.953906863s) [upload=12702]: Waiting for error group managing segments... Aug 7 21:58:56.209900221 (12h19m17.953957082s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:58:56.209905576 (12h19m17.953962423s) [upload=12702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:58:56.209940267 (12h19m17.953997127s) [upload=12702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:58:56.605688962 (12h19m18.349745811s) [upload=12702]: Done uploading segments... Aug 7 21:58:56.605708466 (12h19m18.349765315s) [upload=12702]: Upload finished. err: <nil> Aug 7 21:59:15.802505476 (12h19m37.546562336s) [upload=12707]: Starting upload Aug 7 21:59:15.802568926 (12h19m37.546625778s) [upload=12707]: Done starting upload Aug 7 21:59:15.802684030 (12h19m37.546740881s) [upload=12707]: Uploading segments... Aug 7 21:59:15.802727059 (12h19m37.546783925s) [upload=12707]: Got next segment. Inline: true Aug 7 21:59:15.802732418 (12h19m37.546789279s) [upload=12707]: Waiting for error group managing segments... Aug 7 21:59:15.802792878 (12h19m37.546849738s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:15.802798596 (12h19m37.546855456s) [upload=12707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:15.802802334 (12h19m37.546859194s) [upload=12707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:15.945145817 (12h19m37.689202664s) [upload=12707]: Done uploading segments... Aug 7 21:59:15.945175582 (12h19m37.689232429s) [upload=12707]: Upload finished. err: <nil> Aug 7 21:59:31.540495488 (12h19m53.284552336s) [upload=12712]: Starting upload Aug 7 21:59:31.540551951 (12h19m53.284608811s) [upload=12712]: Done starting upload Aug 7 21:59:31.540628497 (12h19m53.284685347s) [upload=12712]: Uploading segments... Aug 7 21:59:31.540671158 (12h19m53.284728027s) [upload=12712]: Got next segment. Inline: true Aug 7 21:59:31.540677279 (12h19m53.284734139s) [upload=12712]: Waiting for error group managing segments... Aug 7 21:59:31.540730297 (12h19m53.284787147s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:59:31.540735969 (12h19m53.284792829s) [upload=12712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:59:31.540739244 (12h19m53.284796105s) [upload=12712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:31.703548635 (12h19m53.447605493s) [upload=12712]: Done uploading segments... Aug 7 21:59:31.703590031 (12h19m53.447646882s) [upload=12712]: Upload finished. err: <nil> Aug 7 22:00:29.248229344 (12h20m50.992286195s) [upload=12728]: Starting upload Aug 7 22:00:29.248281332 (12h20m50.992338188s) [upload=12728]: Done starting upload Aug 7 22:00:29.248374886 (12h20m50.992431738s) [upload=12728]: Uploading segments... Aug 7 22:00:29.248452896 (12h20m50.992509768s) [upload=12728]: Got next segment. Inline: true Aug 7 22:00:29.248460057 (12h20m50.992516919s) [upload=12728]: Waiting for error group managing segments... Aug 7 22:00:29.248538087 (12h20m50.992594934s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:29.248545868 (12h20m50.992602715s) [upload=12728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:29.248549647 (12h20m50.992606513s) [upload=12728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:29.574503821 (12h20m51.318560681s) [upload=12728]: Done uploading segments... Aug 7 22:00:29.574527802 (12h20m51.318584666s) [upload=12728]: Upload finished. err: <nil> Aug 7 22:01:13.393836672 (12h21m35.137893523s) [upload=12740]: Starting upload Aug 7 22:01:13.393888179 (12h21m35.137945042s) [upload=12740]: Done starting upload Aug 7 22:01:13.394016213 (12h21m35.138073077s) [upload=12740]: Uploading segments... Aug 7 22:01:13.394161981 (12h21m35.13821896s) [upload=12740]: Got next segment. Inline: true Aug 7 22:01:13.394170576 (12h21m35.138227445s) [upload=12740]: Waiting for error group managing segments... Aug 7 22:01:13.394246764 (12h21m35.13830363s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:13.394254537 (12h21m35.138311402s) [upload=12740]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:13.394274292 (12h21m35.138331152s) [upload=12740]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:13.707894690 (12h21m35.451951555s) [upload=12740]: Done uploading segments... Aug 7 22:01:13.707926477 (12h21m35.451983343s) [upload=12740]: Upload finished. err: <nil> Aug 7 22:01:29.783138431 (12h21m51.527195291s) [upload=12746]: Starting upload Aug 7 22:01:29.783212111 (12h21m51.527268976s) [upload=12746]: Done starting upload Aug 7 22:01:29.783446600 (12h21m51.52750345s) [upload=12746]: Uploading segments... Aug 7 22:01:29.783527384 (12h21m51.527584254s) [upload=12746]: Got next segment. Inline: true Aug 7 22:01:29.783533979 (12h21m51.52759083s) [upload=12746]: Waiting for error group managing segments... Aug 7 22:01:29.783596800 (12h21m51.527653652s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:29.783605977 (12h21m51.527662827s) [upload=12746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:29.783609731 (12h21m51.52766658s) [upload=12746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:29.966350494 (12h21m51.710407342s) [upload=12746]: Done uploading segments... Aug 7 22:01:29.966369237 (12h21m51.710426083s) [upload=12746]: Upload finished. err: <nil> Aug 7 22:01:45.372266942 (12h22m7.116323788s) [upload=12750]: Starting upload Aug 7 22:01:45.372314023 (12h22m7.116370883s) [upload=12750]: Done starting upload Aug 7 22:01:45.372374655 (12h22m7.116431503s) [upload=12750]: Uploading segments... Aug 7 22:01:45.372417267 (12h22m7.116474136s) [upload=12750]: Got next segment. Inline: true Aug 7 22:01:45.372422121 (12h22m7.116478981s) [upload=12750]: Waiting for error group managing segments... Aug 7 22:01:45.372461831 (12h22m7.116518692s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:45.372466235 (12h22m7.116523094s) [upload=12750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:45.372468880 (12h22m7.11652574s) [upload=12750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:45.597843459 (12h22m7.341900345s) [upload=12750]: Done uploading segments... Aug 7 22:01:45.597877236 (12h22m7.341934095s) [upload=12750]: Upload finished. err: <nil> Aug 7 22:02:22.792150325 (12h22m44.53620717s) [upload=12764]: Starting upload Aug 7 22:02:22.792208195 (12h22m44.536265055s) [upload=12764]: Done starting upload Aug 7 22:02:22.792266980 (12h22m44.536323827s) [upload=12764]: Uploading segments... Aug 7 22:02:22.792307216 (12h22m44.536364085s) [upload=12764]: Got next segment. Inline: true Aug 7 22:02:22.792311926 (12h22m44.536368786s) [upload=12764]: Waiting for error group managing segments... Aug 7 22:02:22.792351265 (12h22m44.536408125s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.792356546 (12h22m44.536413408s) [upload=12764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.792359208 (12h22m44.536416068s) [upload=12764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:23.139246613 (12h22m44.883303464s) [upload=12764]: Done uploading segments... Aug 7 22:02:23.139265320 (12h22m44.883322173s) [upload=12764]: Upload finished. err: <nil> Duration: 1.05663205s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)