Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
13 out of 60 is failed (threshold=3) [..............................................XX.XXXXXXXXXXX] duration:
30.001068493s
FAIL (2025-08-07 21:22:03 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 627 (11h36m22.311152478s) [upload=12047]: Uploading segments... Aug 7 21:16:00.567233167 (11h36m22.311290039s) [upload=12047]: Got next segment. Inline: true Aug 7 21:16:00.567240534 (11h36m22.311297398s) [upload=12047]: Waiting for error group managing segments... Aug 7 21:16:00.567303473 (11h36m22.311360333s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:00.567314947 (11h36m22.311371807s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:00.567322295 (11h36m22.311379156s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:30.298703441 (11h36m52.0427603s) [upload=12047]: Done uploading segments... Aug 7 21:16:30.298732421 (11h36m52.04278928s) [upload=12047]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:41.168277425 (11h37m2.912334286s) [upload=12055]: Starting upload Aug 7 21:16:41.168334314 (11h37m2.912391183s) [upload=12055]: Done starting upload Aug 7 21:16:41.168407611 (11h37m2.912464477s) [upload=12055]: Uploading segments... Aug 7 21:16:41.168464727 (11h37m2.912521594s) [upload=12055]: Got next segment. Inline: true Aug 7 21:16:41.168470534 (11h37m2.912527404s) [upload=12055]: Waiting for error group managing segments... Aug 7 21:16:41.168520756 (11h37m2.912577617s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:41.168526439 (11h37m2.912583303s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:41.168529826 (11h37m2.91258669s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:10.898488630 (11h37m32.642545483s) [upload=12055]: Done uploading segments... Aug 7 21:17:10.898512884 (11h37m32.642569743s) [upload=12055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:11.314708421 (11h37m33.058765271s) [upload=12064]: Starting upload Aug 7 21:17:11.314772571 (11h37m33.058829423s) [upload=12064]: Done starting upload Aug 7 21:17:11.314846098 (11h37m33.05890295s) [upload=12064]: Uploading segments... Aug 7 21:17:11.314935220 (11h37m33.058992089s) [upload=12064]: Got next segment. Inline: true Aug 7 21:17:11.314940559 (11h37m33.058997419s) [upload=12064]: Waiting for error group managing segments... Aug 7 21:17:11.314984274 (11h37m33.059041125s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:11.314989343 (11h37m33.059046203s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:11.314992536 (11h37m33.059049396s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:41.043046082 (11h38m2.787102951s) [upload=12064]: Done uploading segments... Aug 7 21:17:41.043076101 (11h38m2.787132966s) [upload=12064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:41.498964594 (11h38m3.243021445s) [upload=12072]: Starting upload Aug 7 21:17:41.499038639 (11h38m3.243095509s) [upload=12072]: Done starting upload Aug 7 21:17:41.499118466 (11h38m3.243175318s) [upload=12072]: Uploading segments... Aug 7 21:17:41.499176424 (11h38m3.243233293s) [upload=12072]: Got next segment. Inline: true Aug 7 21:17:41.499181917 (11h38m3.243238777s) [upload=12072]: Waiting for error group managing segments... Aug 7 21:17:41.499236606 (11h38m3.243293472s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:41.499242465 (11h38m3.24329933s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:41.499245854 (11h38m3.243302721s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:11.231025141 (11h38m32.97508201s) [upload=12072]: Done uploading segments... Aug 7 21:18:11.231066796 (11h38m32.975123655s) [upload=12072]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:11.675991066 (11h38m33.420047932s) [upload=12078]: Starting upload Aug 7 21:18:11.676081621 (11h38m33.42013848s) [upload=12078]: Done starting upload Aug 7 21:18:11.676172293 (11h38m33.420229154s) [upload=12078]: Uploading segments... Aug 7 21:18:11.676235479 (11h38m33.420292347s) [upload=12078]: Got next segment. Inline: true Aug 7 21:18:11.676241644 (11h38m33.420298508s) [upload=12078]: Waiting for error group managing segments... Aug 7 21:18:11.676304521 (11h38m33.420361386s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:18:11.676310489 (11h38m33.420367352s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:11.676314037 (11h38m33.420370906s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:41.388891420 (11h39m3.132948271s) [upload=12078]: Done uploading segments... Aug 7 21:18:41.388921897 (11h39m3.132978749s) [upload=12078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:53.498741452 (11h39m15.242798302s) [upload=12086]: Starting upload Aug 7 21:18:53.498798950 (11h39m15.242855819s) [upload=12086]: Done starting upload Aug 7 21:18:53.498860950 (11h39m15.242917802s) [upload=12086]: Uploading segments... Aug 7 21:18:53.498901325 (11h39m15.242958195s) [upload=12086]: Got next segment. Inline: true Aug 7 21:18:53.498908406 (11h39m15.242965267s) [upload=12086]: Waiting for error group managing segments... Aug 7 21:18:53.498951359 (11h39m15.24300821s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:18:53.498956808 (11h39m15.243013668s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:53.498960205 (11h39m15.243017065s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:19:23.226932815 (11h39m44.970989679s) [upload=12086]: Done uploading segments... Aug 7 21:19:23.226954988 (11h39m44.971011848s) [upload=12086]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:19:23.633524671 (11h39m45.37758152s) [upload=12093]: Starting upload Aug 7 21:19:23.633599963 (11h39m45.377656832s) [upload=12093]: Done starting upload Aug 7 21:19:23.633677463 (11h39m45.377734313s) [upload=12093]: Uploading segments... Aug 7 21:19:23.633732206 (11h39m45.377789075s) [upload=12093]: Got next segment. Inline: true Aug 7 21:19:23.633737928 (11h39m45.377794778s) [upload=12093]: Waiting for error group managing segments... Aug 7 21:19:23.633790184 (11h39m45.377847035s) [upload=12093]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:19:23.633795686 (11h39m45.377852546s) [upload=12093]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:19:23.633801264 (11h39m45.377858124s) [upload=12093]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:19:53.363781250 (11h40m15.107838111s) [upload=12093]: Done uploading segments... Aug 7 21:19:53.363801573 (11h40m15.107858434s) [upload=12093]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:24.226894126 (11h40m45.970950987s) [upload=12106]: Starting upload Aug 7 21:20:24.226954589 (11h40m45.971011441s) [upload=12106]: Done starting upload Aug 7 21:20:24.227046095 (11h40m45.971102945s) [upload=12106]: Uploading segments... Aug 7 21:20:24.227086861 (11h40m45.97114373s) [upload=12106]: Got next segment. Inline: true Aug 7 21:20:24.227092422 (11h40m45.971149283s) [upload=12106]: Waiting for error group managing segments... Aug 7 21:20:24.227144188 (11h40m45.971201039s) [upload=12106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:20:24.227149758 (11h40m45.971206619s) [upload=12106]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:24.227153486 (11h40m45.971210345s) [upload=12106]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:20:53.942184977 (11h41m15.68624183s) [upload=12106]: Done uploading segments... Aug 7 21:20:53.942206917 (11h41m15.686263767s) [upload=12106]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:03.619018325 (11h41m25.363075197s) [upload=12116]: Starting upload Aug 7 21:21:03.619120575 (11h41m25.363177439s) [upload=12116]: Done starting upload Aug 7 21:21:03.619209054 (11h41m25.363265908s) [upload=12116]: Uploading segments... Aug 7 21:21:03.619303607 (11h41m25.363360476s) [upload=12116]: Got next segment. Inline: true Aug 7 21:21:03.619310139 (11h41m25.363366989s) [upload=12116]: Waiting for error group managing segments... Aug 7 21:21:03.619383781 (11h41m25.363440632s) [upload=12116]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:21:03.619390203 (11h41m25.363447055s) [upload=12116]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:03.619504810 (11h41m25.363561678s) [upload=12116]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:21:33.340185414 (11h41m55.084242267s) [upload=12116]: Done uploading segments... Aug 7 21:21:33.340212992 (11h41m55.08426986s) [upload=12116]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:33.783052975 (11h41m55.527109844s) [upload=12126]: Starting upload Aug 7 21:21:33.783141523 (11h41m55.527198395s) [upload=12126]: Done starting upload Aug 7 21:21:33.783231997 (11h41m55.527288844s) [upload=12126]: Uploading segments... Aug 7 21:21:33.783290191 (11h41m55.527347062s) [upload=12126]: Got next segment. Inline: true Aug 7 21:21:33.783297437 (11h41m55.527354312s) [upload=12126]: Waiting for error group managing segments... Aug 7 21:21:33.783352192 (11h41m55.527409054s) [upload=12126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:21:33.783359317 (11h41m55.527416182s) [upload=12126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:33.783363849 (11h41m55.527420713s) [upload=12126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:22:03.508585017 (11h42m25.252641871s) [upload=12126]: Done uploading segments... Aug 7 21:22:03.508625142 (11h42m25.252681994s) [upload=12126]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.00108838s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration:
1m0.00120098s
FAIL (2025-08-07 21:21:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001217846s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [.............................................X..XXXXXXXXXXXX] duration:
30.000834119s
FAIL (2025-08-07 21:21:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000855557s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
30.001260612s
FAIL (2025-08-07 21:21:46 UTC)
orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.001288706s
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration:
332.655645ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 21:21:43 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 332.68261ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............................X......................X.......] duration:
2.641090682s
FAIL (2025-08-07 21:21:42 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
oad finished. err: <nil>
Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload
Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload
Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments...
Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false
Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment...
Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment.
Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment
Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments...
Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces.
Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments...
Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil>
Duration: 2.641145329s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
12 out of 60 is failed (threshold=3) [...............................................XX.XXXXXXXXXX] duration:
30.000775179s
FAIL (2025-08-07 21:21:33 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 5806 (11h35m52.153952669s) [upload=12036]: Uploading segments... Aug 7 21:15:30.409994454 (11h35m52.154051324s) [upload=12036]: Got next segment. Inline: true Aug 7 21:15:30.410001467 (11h35m52.154058337s) [upload=12036]: Waiting for error group managing segments... Aug 7 21:15:30.410069055 (11h35m52.154125913s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:15:30.410077097 (11h35m52.154133959s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:30.410082368 (11h35m52.154139256s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:00.139341290 (11h36m21.883398189s) [upload=12036]: Done uploading segments... Aug 7 21:16:00.139580695 (11h36m21.88363756s) [upload=12036]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:00.566884400 (11h36m22.310941251s) [upload=12047]: Starting upload Aug 7 21:16:00.566993382 (11h36m22.311050244s) [upload=12047]: Done starting upload Aug 7 21:16:00.567095627 (11h36m22.311152478s) [upload=12047]: Uploading segments... Aug 7 21:16:00.567233167 (11h36m22.311290039s) [upload=12047]: Got next segment. Inline: true Aug 7 21:16:00.567240534 (11h36m22.311297398s) [upload=12047]: Waiting for error group managing segments... Aug 7 21:16:00.567303473 (11h36m22.311360333s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:00.567314947 (11h36m22.311371807s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:00.567322295 (11h36m22.311379156s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:30.298703441 (11h36m52.0427603s) [upload=12047]: Done uploading segments... Aug 7 21:16:30.298732421 (11h36m52.04278928s) [upload=12047]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:41.168277425 (11h37m2.912334286s) [upload=12055]: Starting upload Aug 7 21:16:41.168334314 (11h37m2.912391183s) [upload=12055]: Done starting upload Aug 7 21:16:41.168407611 (11h37m2.912464477s) [upload=12055]: Uploading segments... Aug 7 21:16:41.168464727 (11h37m2.912521594s) [upload=12055]: Got next segment. Inline: true Aug 7 21:16:41.168470534 (11h37m2.912527404s) [upload=12055]: Waiting for error group managing segments... Aug 7 21:16:41.168520756 (11h37m2.912577617s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:41.168526439 (11h37m2.912583303s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:41.168529826 (11h37m2.91258669s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:10.898488630 (11h37m32.642545483s) [upload=12055]: Done uploading segments... Aug 7 21:17:10.898512884 (11h37m32.642569743s) [upload=12055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:11.314708421 (11h37m33.058765271s) [upload=12064]: Starting upload Aug 7 21:17:11.314772571 (11h37m33.058829423s) [upload=12064]: Done starting upload Aug 7 21:17:11.314846098 (11h37m33.05890295s) [upload=12064]: Uploading segments... Aug 7 21:17:11.314935220 (11h37m33.058992089s) [upload=12064]: Got next segment. Inline: true Aug 7 21:17:11.314940559 (11h37m33.058997419s) [upload=12064]: Waiting for error group managing segments... Aug 7 21:17:11.314984274 (11h37m33.059041125s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:11.314989343 (11h37m33.059046203s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:11.314992536 (11h37m33.059049396s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:41.043046082 (11h38m2.787102951s) [upload=12064]: Done uploading segments... Aug 7 21:17:41.043076101 (11h38m2.787132966s) [upload=12064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:41.498964594 (11h38m3.243021445s) [upload=12072]: Starting upload Aug 7 21:17:41.499038639 (11h38m3.243095509s) [upload=12072]: Done starting upload Aug 7 21:17:41.499118466 (11h38m3.243175318s) [upload=12072]: Uploading segments... Aug 7 21:17:41.499176424 (11h38m3.243233293s) [upload=12072]: Got next segment. Inline: true Aug 7 21:17:41.499181917 (11h38m3.243238777s) [upload=12072]: Waiting for error group managing segments... Aug 7 21:17:41.499236606 (11h38m3.243293472s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:41.499242465 (11h38m3.24329933s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:41.499245854 (11h38m3.243302721s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:11.231025141 (11h38m32.97508201s) [upload=12072]: Done uploading segments... Aug 7 21:18:11.231066796 (11h38m32.975123655s) [upload=12072]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:11.675991066 (11h38m33.420047932s) [upload=12078]: Starting upload Aug 7 21:18:11.676081621 (11h38m33.42013848s) [upload=12078]: Done starting upload Aug 7 21:18:11.676172293 (11h38m33.420229154s) [upload=12078]: Uploading segments... Aug 7 21:18:11.676235479 (11h38m33.420292347s) [upload=12078]: Got next segment. Inline: true Aug 7 21:18:11.676241644 (11h38m33.420298508s) [upload=12078]: Waiting for error group managing segments... Aug 7 21:18:11.676304521 (11h38m33.420361386s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:18:11.676310489 (11h38m33.420367352s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:11.676314037 (11h38m33.420370906s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:41.388891420 (11h39m3.132948271s) [upload=12078]: Done uploading segments... Aug 7 21:18:41.388921897 (11h39m3.132978749s) [upload=12078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:53.498741452 (11h39m15.242798302s) [upload=12086]: Starting upload Aug 7 21:18:53.498798950 (11h39m15.242855819s) [upload=12086]: Done starting upload Aug 7 21:18:53.498860950 (11h39m15.242917802s) [upload=12086]: Uploading segments... Aug 7 21:18:53.498901325 (11h39m15.242958195s) [upload=12086]: Got next segment. Inline: true Aug 7 21:18:53.498908406 (11h39m15.242965267s) [upload=12086]: Waiting for error group managing segments... Aug 7 21:18:53.498951359 (11h39m15.24300821s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:18:53.498956808 (11h39m15.243013668s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:53.498960205 (11h39m15.243017065s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:19:23.226932815 (11h39m44.970989679s) [upload=12086]: Done uploading segments... Aug 7 21:19:23.226954988 (11h39m44.971011848s) [upload=12086]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:19:23.633524671 (11h39m45.37758152s) [upload=12093]: Starting upload Aug 7 21:19:23.633599963 (11h39m45.377656832s) [upload=12093]: Done starting upload Aug 7 21:19:23.633677463 (11h39m45.377734313s) [upload=12093]: Uploading segments... Aug 7 21:19:23.633732206 (11h39m45.377789075s) [upload=12093]: Got next segment. Inline: true Aug 7 21:19:23.633737928 (11h39m45.377794778s) [upload=12093]: Waiting for error group managing segments... Aug 7 21:19:23.633790184 (11h39m45.377847035s) [upload=12093]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:19:23.633795686 (11h39m45.377852546s) [upload=12093]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:19:23.633801264 (11h39m45.377858124s) [upload=12093]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:19:53.363781250 (11h40m15.107838111s) [upload=12093]: Done uploading segments... Aug 7 21:19:53.363801573 (11h40m15.107858434s) [upload=12093]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:24.226894126 (11h40m45.970950987s) [upload=12106]: Starting upload Aug 7 21:20:24.226954589 (11h40m45.971011441s) [upload=12106]: Done starting upload Aug 7 21:20:24.227046095 (11h40m45.971102945s) [upload=12106]: Uploading segments... Aug 7 21:20:24.227086861 (11h40m45.97114373s) [upload=12106]: Got next segment. Inline: true Aug 7 21:20:24.227092422 (11h40m45.971149283s) [upload=12106]: Waiting for error group managing segments... Aug 7 21:20:24.227144188 (11h40m45.971201039s) [upload=12106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:20:24.227149758 (11h40m45.971206619s) [upload=12106]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:24.227153486 (11h40m45.971210345s) [upload=12106]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:20:53.942184977 (11h41m15.68624183s) [upload=12106]: Done uploading segments... Aug 7 21:20:53.942206917 (11h41m15.686263767s) [upload=12106]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:03.619018325 (11h41m25.363075197s) [upload=12116]: Starting upload Aug 7 21:21:03.619120575 (11h41m25.363177439s) [upload=12116]: Done starting upload Aug 7 21:21:03.619209054 (11h41m25.363265908s) [upload=12116]: Uploading segments... Aug 7 21:21:03.619303607 (11h41m25.363360476s) [upload=12116]: Got next segment. Inline: true Aug 7 21:21:03.619310139 (11h41m25.363366989s) [upload=12116]: Waiting for error group managing segments... Aug 7 21:21:03.619383781 (11h41m25.363440632s) [upload=12116]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:21:03.619390203 (11h41m25.363447055s) [upload=12116]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:03.619504810 (11h41m25.363561678s) [upload=12116]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:21:33.340185414 (11h41m55.084242267s) [upload=12116]: Done uploading segments... Aug 7 21:21:33.340212992 (11h41m55.08426986s) [upload=12116]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.000830638s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..X.......X.................................................] duration:
863.262491ms
FAIL (2025-08-07 21:21:33 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
68895 (10h54m20.003725754s) [upload=11347]: Done uploading segments...
Aug 7 20:33:58.259694049 (10h54m20.00375091s) [upload=11347]: Upload finished. err: <nil>
Aug 7 20:34:42.291300370 (10h55m4.035357231s) [upload=11357]: Starting upload
Aug 7 20:34:42.291392440 (10h55m4.035449315s) [upload=11357]: Done starting upload
Aug 7 20:34:42.291505939 (10h55m4.035562788s) [upload=11357]: Uploading segments...
Aug 7 20:34:42.291587798 (10h55m4.035644667s) [upload=11357]: Got next segment. Inline: true
Aug 7 20:34:42.291610428 (10h55m4.035667289s) [upload=11357]: Waiting for error group managing segments...
Aug 7 20:34:42.291660078 (10h55m4.035716927s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:34:42.291665407 (10h55m4.035722257s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:34:42.291668818 (10h55m4.035725679s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:34:42.445169337 (10h55m4.189226185s) [upload=11357]: Done uploading segments...
Aug 7 20:34:42.445187662 (10h55m4.189244508s) [upload=11357]: Upload finished. err: <nil>
Aug 7 20:35:56.993979713 (10h56m18.738036563s) [upload=11378]: Starting upload
Aug 7 20:35:56.994042734 (10h56m18.738099594s) [upload=11378]: Done starting upload
Aug 7 20:35:56.994133564 (10h56m18.738190418s) [upload=11378]: Uploading segments...
Aug 7 20:35:56.994188836 (10h56m18.738245705s) [upload=11378]: Got next segment. Inline: true
Aug 7 20:35:56.994194253 (10h56m18.738251113s) [upload=11378]: Waiting for error group managing segments...
Aug 7 20:35:56.994255059 (10h56m18.738311921s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:35:56.994261721 (10h56m18.738318586s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:35:56.994266228 (10h56m18.73832309s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:35:57.196915243 (10h56m18.940972094s) [upload=11378]: Done uploading segments...
Aug 7 20:35:57.196933580 (10h56m18.940990431s) [upload=11378]: Upload finished. err: <nil>
Aug 7 20:36:48.383926585 (10h57m10.127983431s) [upload=11392]: Starting upload
Aug 7 20:36:48.383977279 (10h57m10.128034139s) [upload=11392]: Done starting upload
Aug 7 20:36:48.384080928 (10h57m10.128137781s) [upload=11392]: Uploading segments...
Aug 7 20:36:48.384124913 (10h57m10.128181781s) [upload=11392]: Got next segment. Inline: true
Aug 7 20:36:48.384130188 (10h57m10.128187037s) [upload=11392]: Waiting for error group managing segments...
Aug 7 20:36:48.384183298 (10h57m10.128240148s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:36:48.384188890 (10h57m10.128245741s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:36:48.384192353 (10h57m10.128249203s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:36:48.597716681 (10h57m10.341773532s) [upload=11392]: Done uploading segments...
Aug 7 20:36:48.597735626 (10h57m10.341792476s) [upload=11392]: Upload finished. err: <nil>
Aug 7 20:37:59.886756865 (10h58m21.630813711s) [upload=11411]: Starting upload
Aug 7 20:37:59.886810662 (10h58m21.630867522s) [upload=11411]: Done starting upload
Aug 7 20:37:59.886870714 (10h58m21.630927561s) [upload=11411]: Uploading segments...
Aug 7 20:37:59.886906763 (10h58m21.630963634s) [upload=11411]: Got next segment. Inline: true
Aug 7 20:37:59.886911413 (10h58m21.630968259s) [upload=11411]: Waiting for error group managing segments...
Aug 7 20:37:59.886948286 (10h58m21.631005132s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:37:59.886952923 (10h58m21.631009769s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:37:59.886957250 (10h58m21.631014096s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:38:00.073123067 (10h58m21.817179918s) [upload=11411]: Done uploading segments...
Aug 7 20:38:00.073142248 (10h58m21.817199097s) [upload=11411]: Upload finished. err: <nil>
Aug 7 20:39:06.192304563 (10h59m27.936361423s) [upload=11429]: Starting upload
Aug 7 20:39:06.192379945 (10h59m27.936436804s) [upload=11429]: Done starting upload
Aug 7 20:39:06.192450925 (10h59m27.936507772s) [upload=11429]: Uploading segments...
Aug 7 20:39:06.192493866 (10h59m27.936550726s) [upload=11429]: Got next segment. Inline: true
Aug 7 20:39:06.192499400 (10h59m27.93655626s) [upload=11429]: Waiting for error group managing segments...
Aug 7 20:39:06.192549058 (10h59m27.936605918s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:39:06.192553658 (10h59m27.936610518s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:39:06.192556283 (10h59m27.936613143s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:39:06.410732058 (10h59m28.154788918s) [upload=11429]: Done uploading segments...
Aug 7 20:39:06.410751072 (10h59m28.154807921s) [upload=11429]: Upload finished. err: <nil>
Aug 7 20:39:38.189437550 (10h59m59.9334944s) [upload=11438]: Starting upload
Aug 7 20:39:38.189494699 (10h59m59.933551565s) [upload=11438]: Done starting upload
Aug 7 20:39:38.189570885 (10h59m59.933627735s) [upload=11438]: Uploading segments...
Aug 7 20:39:38.189613828 (10h59m59.933670698s) [upload=11438]: Got next segment. Inline: true
Aug 7 20:39:38.189618898 (10h59m59.933675758s) [upload=11438]: Waiting for error group managing segments...
Aug 7 20:39:38.189671547 (10h59m59.933728396s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:39:38.189676923 (10h59m59.933733774s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:39:38.189680363 (10h59m59.933737214s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:39:38.339370287 (11h0m0.083427138s) [upload=11438]: Done uploading segments...
Aug 7 20:39:38.339394965 (11h0m0.083451822s) [upload=11438]: Upload finished. err: <nil>
Aug 7 20:40:13.049368895 (11h0m34.793425745s) [upload=11448]: Starting upload
Aug 7 20:40:13.049412890 (11h0m34.793469743s) [upload=11448]: Done starting upload
Aug 7 20:40:13.049490173 (11h0m34.793547027s) [upload=11448]: Uploading segments...
Aug 7 20:40:13.049531553 (11h0m34.793588423s) [upload=11448]: Got next segment. Inline: true
Aug 7 20:40:13.049536305 (11h0m34.79359316s) [upload=11448]: Waiting for error group managing segments...
Aug 7 20:40:13.049606089 (11h0m34.793662937s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:40:13.049611863 (11h0m34.793668715s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:40:13.049615487 (11h0m34.793672338s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:40:13.251485869 (11h0m34.995542721s) [upload=11448]: Done uploading segments...
Aug 7 20:40:13.251504252 (11h0m34.995561102s) [upload=11448]: Upload finished. err: <nil>
Aug 7 20:41:01.300323709 (11h1m23.044380561s) [upload=11462]: Starting upload
Aug 7 20:41:01.300386685 (11h1m23.044443554s) [upload=11462]: Done starting upload
Aug 7 20:41:01.300471424 (11h1m23.044528277s) [upload=11462]: Uploading segments...
Aug 7 20:41:01.300528867 (11h1m23.044585736s) [upload=11462]: Got next segment. Inline: true
Aug 7 20:41:01.300534426 (11h1m23.044591286s) [upload=11462]: Waiting for error group managing segments...
Aug 7 20:41:01.300592418 (11h1m23.044649268s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:01.300598189 (11h1m23.04465505s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:01.300601616 (11h1m23.044658476s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:01.448083138 (11h1m23.192139989s) [upload=11462]: Done uploading segments...
Aug 7 20:41:01.448102791 (11h1m23.192159644s) [upload=11462]: Upload finished. err: <nil>
Aug 7 20:41:25.522931110 (11h1m47.266987978s) [upload=11468]: Starting upload
Aug 7 20:41:25.523039051 (11h1m47.267095924s) [upload=11468]: Done starting upload
Aug 7 20:41:25.523181759 (11h1m47.267238621s) [upload=11468]: Uploading segments...
Aug 7 20:41:25.523403073 (11h1m47.267459944s) [upload=11468]: Got next segment. Inline: true
Aug 7 20:41:25.523413059 (11h1m47.26746993s) [upload=11468]: Waiting for error group managing segments...
Aug 7 20:41:25.523492519 (11h1m47.267549389s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:25.523498700 (11h1m47.267555566s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:25.523502761 (11h1m47.267559629s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:25.840530122 (11h1m47.584586983s) [upload=11468]: Done uploading segments...
Aug 7 20:41:25.840544406 (11h1m47.584601253s) [upload=11468]: Upload finished. err: <nil>
Aug 7 20:41:49.037046771 (11h2m10.781103621s) [upload=11474]: Starting upload
Aug 7 20:41:49.037108376 (11h2m10.781165228s) [upload=11474]: Done starting upload
Aug 7 20:41:49.037181045 (11h2m10.781237895s) [upload=11474]: Uploading segments...
Aug 7 20:41:49.037238734 (11h2m10.781295606s) [upload=11474]: Got next segment. Inline: true
Aug 7 20:41:49.037244163 (11h2m10.781301028s) [upload=11474]: Waiting for error group managing segments...
Aug 7 20:41:49.037309765 (11h2m10.781366628s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:49.037314702 (11h2m10.781371562s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:49.037317447 (11h2m10.781374307s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:49.205260584 (11h2m10.949317434s) [upload=11474]: Done uploading segments...
Aug 7 20:41:49.205278299 (11h2m10.94933515s) [upload=11474]: Upload finished. err: <nil>
Duration: 863.299141ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration:
30.00108338s
FAIL (2025-08-07 21:21:33 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 30.00113549s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
12 out of 60 is failed (threshold=3) [..............................................X..XXXXXXXXXXX] duration:
30.000640826s
FAIL (2025-08-07 21:21:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000656383s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration:
30.002715631s
FAIL (2025-08-07 21:21:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.002760305s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
4 out of 60 is failed (threshold=3) [........................................................XXXX] duration:
30.001460116s
FAIL (2025-08-07 21:21:12 UTC)
orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.001486428s
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [......................................................XX....] duration:
323.42447ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 21:21:11 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 323.465039ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
6 out of 60 is failed (threshold=3) [.....................................................X.XXXXX] duration:
1m0.001420227s
FAIL (2025-08-07 21:21:04 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ing upload Aug 7 21:11:16.700090349 (11h31m38.44414722s) [upload=11969]: Done starting upload Aug 7 21:11:16.700171865 (11h31m38.444228712s) [upload=11969]: Uploading segments... Aug 7 21:11:16.700229408 (11h31m38.444286283s) [upload=11969]: Got next segment. Inline: true Aug 7 21:11:16.700236785 (11h31m38.444293651s) [upload=11969]: Waiting for error group managing segments... Aug 7 21:11:16.700303583 (11h31m38.444360444s) [upload=11969]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:11:16.700311256 (11h31m38.444368121s) [upload=11969]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:16.700316036 (11h31m38.444372904s) [upload=11969]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:11:16.845168013 (11h31m38.589224864s) [upload=11969]: Done uploading segments... Aug 7 21:11:16.845187939 (11h31m38.5892448s) [upload=11969]: Upload finished. err: <nil> Aug 7 21:12:04.147595666 (11h32m25.891652543s) [upload=11983]: Starting upload Aug 7 21:12:04.147723257 (11h32m25.891780138s) [upload=11983]: Done starting upload Aug 7 21:12:04.147862477 (11h32m25.891919358s) [upload=11983]: Uploading segments... Aug 7 21:12:04.147924428 (11h32m25.891981303s) [upload=11983]: Got next segment. Inline: true Aug 7 21:12:04.147931028 (11h32m25.89198791s) [upload=11983]: Waiting for error group managing segments... Aug 7 21:12:04.147990094 (11h32m25.892046969s) [upload=11983]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:12:04.147996570 (11h32m25.892053447s) [upload=11983]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:04.148003467 (11h32m25.892060346s) [upload=11983]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:12:04.368229522 (11h32m26.112286373s) [upload=11983]: Done uploading segments... Aug 7 21:12:04.368249120 (11h32m26.11230597s) [upload=11983]: Upload finished. err: <nil> Aug 7 21:12:47.792015773 (11h33m9.536072646s) [upload=11992]: Starting upload Aug 7 21:12:47.792127865 (11h33m9.536184732s) [upload=11992]: Done starting upload Aug 7 21:12:47.792248444 (11h33m9.536305307s) [upload=11992]: Uploading segments... Aug 7 21:12:47.792351067 (11h33m9.536407934s) [upload=11992]: Got next segment. Inline: true Aug 7 21:12:47.792359617 (11h33m9.536416486s) [upload=11992]: Waiting for error group managing segments... Aug 7 21:12:47.792414618 (11h33m9.536471489s) [upload=11992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:12:47.792420367 (11h33m9.536477238s) [upload=11992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:47.792423893 (11h33m9.536480763s) [upload=11992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:12:47.975624008 (11h33m9.71968088s) [upload=11992]: Done uploading segments... Aug 7 21:12:47.975648860 (11h33m9.71970573s) [upload=11992]: Upload finished. err: <nil> Aug 7 21:14:01.226859239 (11h34m22.970916111s) [upload=12013]: Starting upload Aug 7 21:14:01.227003646 (11h34m22.971060518s) [upload=12013]: Done starting upload Aug 7 21:14:01.227141194 (11h34m22.971198047s) [upload=12013]: Uploading segments... Aug 7 21:14:01.227219219 (11h34m22.971276084s) [upload=12013]: Got next segment. Inline: true Aug 7 21:14:01.227227170 (11h34m22.971284041s) [upload=12013]: Waiting for error group managing segments... Aug 7 21:14:01.227295298 (11h34m22.971352162s) [upload=12013]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:14:01.227301580 (11h34m22.971358443s) [upload=12013]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:01.227305300 (11h34m22.971362163s) [upload=12013]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:15:00.953618154 (11h35m22.697675004s) [upload=12013]: Done uploading segments... Aug 7 21:15:00.953640566 (11h35m22.697697426s) [upload=12013]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:01.394116473 (11h35m23.138173339s) [upload=12030]: Starting upload Aug 7 21:15:01.394242598 (11h35m23.138299459s) [upload=12030]: Done starting upload Aug 7 21:15:01.394373467 (11h35m23.138430318s) [upload=12030]: Uploading segments... Aug 7 21:15:01.394459055 (11h35m23.138515928s) [upload=12030]: Got next segment. Inline: true Aug 7 21:15:01.394467128 (11h35m23.138523996s) [upload=12030]: Waiting for error group managing segments... Aug 7 21:15:01.394548041 (11h35m23.138604915s) [upload=12030]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:15:01.394554816 (11h35m23.13861169s) [upload=12030]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:01.394559518 (11h35m23.138616392s) [upload=12030]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:15:03.584668408 (11h35m25.328725286s) [upload=12030]: Done uploading segments... Aug 7 21:15:03.584713689 (11h35m25.32877056s) [upload=12030]: Upload finished. err: <nil> Aug 7 21:15:54.636790163 (11h36m16.380847013s) [upload=12044]: Starting upload Aug 7 21:15:54.636877707 (11h36m16.380934638s) [upload=12044]: Done starting upload Aug 7 21:15:54.636936350 (11h36m16.3809932s) [upload=12044]: Uploading segments... Aug 7 21:15:54.636993824 (11h36m16.381050685s) [upload=12044]: Got next segment. Inline: true Aug 7 21:15:54.636999384 (11h36m16.381056244s) [upload=12044]: Waiting for error group managing segments... Aug 7 21:15:54.637044877 (11h36m16.381101727s) [upload=12044]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:15:54.637050662 (11h36m16.381107522s) [upload=12044]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:54.637053905 (11h36m16.381110766s) [upload=12044]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:54.359175610 (11h37m16.103232476s) [upload=12044]: Done uploading segments... Aug 7 21:16:54.359203156 (11h37m16.103260018s) [upload=12044]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:54.796260076 (11h37m16.54031694s) [upload=12060]: Starting upload Aug 7 21:16:54.796405798 (11h37m16.540462658s) [upload=12060]: Done starting upload Aug 7 21:16:54.796514557 (11h37m16.540571407s) [upload=12060]: Uploading segments... Aug 7 21:16:54.796581012 (11h37m16.540637875s) [upload=12060]: Got next segment. Inline: true Aug 7 21:16:54.796586298 (11h37m16.540643158s) [upload=12060]: Waiting for error group managing segments... Aug 7 21:16:54.796657533 (11h37m16.540714383s) [upload=12060]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:54.796663030 (11h37m16.540719894s) [upload=12060]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:54.796669424 (11h37m16.540726289s) [upload=12060]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:54.519327140 (11h38m16.263384004s) [upload=12060]: Done uploading segments... Aug 7 21:17:54.519368022 (11h38m16.263424882s) [upload=12060]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:54.944498267 (11h38m16.688555136s) [upload=12074]: Starting upload Aug 7 21:17:54.944617406 (11h38m16.688674267s) [upload=12074]: Done starting upload Aug 7 21:17:54.944695430 (11h38m16.688752285s) [upload=12074]: Uploading segments... Aug 7 21:17:54.944771865 (11h38m16.688828736s) [upload=12074]: Got next segment. Inline: true Aug 7 21:17:54.944781252 (11h38m16.688838123s) [upload=12074]: Waiting for error group managing segments... Aug 7 21:17:54.944841042 (11h38m16.688897916s) [upload=12074]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:54.944848212 (11h38m16.688905079s) [upload=12074]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:54.944851963 (11h38m16.688908827s) [upload=12074]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:54.673864467 (11h39m16.417921315s) [upload=12074]: Done uploading segments... Aug 7 21:18:54.673885859 (11h39m16.417942708s) [upload=12074]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:55.119912759 (11h39m16.863969619s) [upload=12087]: Starting upload Aug 7 21:18:55.120013170 (11h39m16.864070018s) [upload=12087]: Done starting upload Aug 7 21:18:55.120094735 (11h39m16.864151596s) [upload=12087]: Uploading segments... Aug 7 21:18:55.120142836 (11h39m16.864199705s) [upload=12087]: Got next segment. Inline: true Aug 7 21:18:55.120147776 (11h39m16.864204637s) [upload=12087]: Waiting for error group managing segments... Aug 7 21:18:55.120201712 (11h39m16.864258573s) [upload=12087]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:18:55.120207674 (11h39m16.864264534s) [upload=12087]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:55.120211321 (11h39m16.864268183s) [upload=12087]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:19:54.844368358 (11h40m16.588425206s) [upload=12087]: Done uploading segments... Aug 7 21:19:54.844395070 (11h40m16.588451926s) [upload=12087]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:04.793025148 (11h40m26.537081997s) [upload=12103]: Starting upload Aug 7 21:20:04.793120317 (11h40m26.53717719s) [upload=12103]: Done starting upload Aug 7 21:20:04.793244612 (11h40m26.537301459s) [upload=12103]: Uploading segments... Aug 7 21:20:04.793331764 (11h40m26.537388638s) [upload=12103]: Got next segment. Inline: true Aug 7 21:20:04.793339748 (11h40m26.537396613s) [upload=12103]: Waiting for error group managing segments... Aug 7 21:20:04.793417981 (11h40m26.537474831s) [upload=12103]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:20:04.793424412 (11h40m26.537481262s) [upload=12103]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:04.793427866 (11h40m26.537484716s) [upload=12103]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:21:04.512178875 (11h41m26.256235737s) [upload=12103]: Done uploading segments... Aug 7 21:21:04.512205338 (11h41m26.256262189s) [upload=12103]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m0.001466421s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration:
1m0.000446268s
FAIL (2025-08-07 21:20:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000468649s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
11 out of 60 is failed (threshold=3) [...............................................X..XXXXXXXXXX] duration:
30.000365374s
FAIL (2025-08-07 21:20:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000388665s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
11 out of 60 is failed (threshold=3) [................................................XX.XXXXXXXXX] duration:
30.000649286s
FAIL (2025-08-07 21:20:53 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ng upload Aug 7 21:14:18.164985120 (11h34m39.909041981s) [upload=12020]: Uploading segments... Aug 7 21:14:18.165076859 (11h34m39.909133729s) [upload=12020]: Got next segment. Inline: true Aug 7 21:14:18.165084880 (11h34m39.909141745s) [upload=12020]: Waiting for error group managing segments... Aug 7 21:14:18.165136947 (11h34m39.909193807s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:14:18.165142336 (11h34m39.909199196s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:18.165145871 (11h34m39.909202731s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:14:18.323816200 (11h34m40.067873054s) [upload=12020]: Done uploading segments... Aug 7 21:14:18.323839479 (11h34m40.067896328s) [upload=12020]: Upload finished. err: <nil> Aug 7 21:15:30.409753622 (11h35m52.153810483s) [upload=12036]: Starting upload Aug 7 21:15:30.409818667 (11h35m52.153875531s) [upload=12036]: Done starting upload Aug 7 21:15:30.409895806 (11h35m52.153952669s) [upload=12036]: Uploading segments... Aug 7 21:15:30.409994454 (11h35m52.154051324s) [upload=12036]: Got next segment. Inline: true Aug 7 21:15:30.410001467 (11h35m52.154058337s) [upload=12036]: Waiting for error group managing segments... Aug 7 21:15:30.410069055 (11h35m52.154125913s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:15:30.410077097 (11h35m52.154133959s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:30.410082368 (11h35m52.154139256s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:00.139341290 (11h36m21.883398189s) [upload=12036]: Done uploading segments... Aug 7 21:16:00.139580695 (11h36m21.88363756s) [upload=12036]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:00.566884400 (11h36m22.310941251s) [upload=12047]: Starting upload Aug 7 21:16:00.566993382 (11h36m22.311050244s) [upload=12047]: Done starting upload Aug 7 21:16:00.567095627 (11h36m22.311152478s) [upload=12047]: Uploading segments... Aug 7 21:16:00.567233167 (11h36m22.311290039s) [upload=12047]: Got next segment. Inline: true Aug 7 21:16:00.567240534 (11h36m22.311297398s) [upload=12047]: Waiting for error group managing segments... Aug 7 21:16:00.567303473 (11h36m22.311360333s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:00.567314947 (11h36m22.311371807s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:00.567322295 (11h36m22.311379156s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:30.298703441 (11h36m52.0427603s) [upload=12047]: Done uploading segments... Aug 7 21:16:30.298732421 (11h36m52.04278928s) [upload=12047]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:41.168277425 (11h37m2.912334286s) [upload=12055]: Starting upload Aug 7 21:16:41.168334314 (11h37m2.912391183s) [upload=12055]: Done starting upload Aug 7 21:16:41.168407611 (11h37m2.912464477s) [upload=12055]: Uploading segments... Aug 7 21:16:41.168464727 (11h37m2.912521594s) [upload=12055]: Got next segment. Inline: true Aug 7 21:16:41.168470534 (11h37m2.912527404s) [upload=12055]: Waiting for error group managing segments... Aug 7 21:16:41.168520756 (11h37m2.912577617s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:41.168526439 (11h37m2.912583303s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:41.168529826 (11h37m2.91258669s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:10.898488630 (11h37m32.642545483s) [upload=12055]: Done uploading segments... Aug 7 21:17:10.898512884 (11h37m32.642569743s) [upload=12055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:11.314708421 (11h37m33.058765271s) [upload=12064]: Starting upload Aug 7 21:17:11.314772571 (11h37m33.058829423s) [upload=12064]: Done starting upload Aug 7 21:17:11.314846098 (11h37m33.05890295s) [upload=12064]: Uploading segments... Aug 7 21:17:11.314935220 (11h37m33.058992089s) [upload=12064]: Got next segment. Inline: true Aug 7 21:17:11.314940559 (11h37m33.058997419s) [upload=12064]: Waiting for error group managing segments... Aug 7 21:17:11.314984274 (11h37m33.059041125s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:11.314989343 (11h37m33.059046203s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:11.314992536 (11h37m33.059049396s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:41.043046082 (11h38m2.787102951s) [upload=12064]: Done uploading segments... Aug 7 21:17:41.043076101 (11h38m2.787132966s) [upload=12064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:41.498964594 (11h38m3.243021445s) [upload=12072]: Starting upload Aug 7 21:17:41.499038639 (11h38m3.243095509s) [upload=12072]: Done starting upload Aug 7 21:17:41.499118466 (11h38m3.243175318s) [upload=12072]: Uploading segments... Aug 7 21:17:41.499176424 (11h38m3.243233293s) [upload=12072]: Got next segment. Inline: true Aug 7 21:17:41.499181917 (11h38m3.243238777s) [upload=12072]: Waiting for error group managing segments... Aug 7 21:17:41.499236606 (11h38m3.243293472s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:41.499242465 (11h38m3.24329933s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:41.499245854 (11h38m3.243302721s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:11.231025141 (11h38m32.97508201s) [upload=12072]: Done uploading segments... Aug 7 21:18:11.231066796 (11h38m32.975123655s) [upload=12072]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:11.675991066 (11h38m33.420047932s) [upload=12078]: Starting upload Aug 7 21:18:11.676081621 (11h38m33.42013848s) [upload=12078]: Done starting upload Aug 7 21:18:11.676172293 (11h38m33.420229154s) [upload=12078]: Uploading segments... Aug 7 21:18:11.676235479 (11h38m33.420292347s) [upload=12078]: Got next segment. Inline: true Aug 7 21:18:11.676241644 (11h38m33.420298508s) [upload=12078]: Waiting for error group managing segments... Aug 7 21:18:11.676304521 (11h38m33.420361386s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:18:11.676310489 (11h38m33.420367352s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:11.676314037 (11h38m33.420370906s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:41.388891420 (11h39m3.132948271s) [upload=12078]: Done uploading segments... Aug 7 21:18:41.388921897 (11h39m3.132978749s) [upload=12078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:53.498741452 (11h39m15.242798302s) [upload=12086]: Starting upload Aug 7 21:18:53.498798950 (11h39m15.242855819s) [upload=12086]: Done starting upload Aug 7 21:18:53.498860950 (11h39m15.242917802s) [upload=12086]: Uploading segments... Aug 7 21:18:53.498901325 (11h39m15.242958195s) [upload=12086]: Got next segment. Inline: true Aug 7 21:18:53.498908406 (11h39m15.242965267s) [upload=12086]: Waiting for error group managing segments... Aug 7 21:18:53.498951359 (11h39m15.24300821s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:18:53.498956808 (11h39m15.243013668s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:53.498960205 (11h39m15.243017065s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:19:23.226932815 (11h39m44.970989679s) [upload=12086]: Done uploading segments... Aug 7 21:19:23.226954988 (11h39m44.971011848s) [upload=12086]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:19:23.633524671 (11h39m45.37758152s) [upload=12093]: Starting upload Aug 7 21:19:23.633599963 (11h39m45.377656832s) [upload=12093]: Done starting upload Aug 7 21:19:23.633677463 (11h39m45.377734313s) [upload=12093]: Uploading segments... Aug 7 21:19:23.633732206 (11h39m45.377789075s) [upload=12093]: Got next segment. Inline: true Aug 7 21:19:23.633737928 (11h39m45.377794778s) [upload=12093]: Waiting for error group managing segments... Aug 7 21:19:23.633790184 (11h39m45.377847035s) [upload=12093]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:19:23.633795686 (11h39m45.377852546s) [upload=12093]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:19:23.633801264 (11h39m45.377858124s) [upload=12093]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:19:53.363781250 (11h40m15.107838111s) [upload=12093]: Done uploading segments... Aug 7 21:19:53.363801573 (11h40m15.107858434s) [upload=12093]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:24.226894126 (11h40m45.970950987s) [upload=12106]: Starting upload Aug 7 21:20:24.226954589 (11h40m45.971011441s) [upload=12106]: Done starting upload Aug 7 21:20:24.227046095 (11h40m45.971102945s) [upload=12106]: Uploading segments... Aug 7 21:20:24.227086861 (11h40m45.97114373s) [upload=12106]: Got next segment. Inline: true Aug 7 21:20:24.227092422 (11h40m45.971149283s) [upload=12106]: Waiting for error group managing segments... Aug 7 21:20:24.227144188 (11h40m45.971201039s) [upload=12106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:20:24.227149758 (11h40m45.971206619s) [upload=12106]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:24.227153486 (11h40m45.971210345s) [upload=12106]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:20:53.942184977 (11h41m15.68624183s) [upload=12106]: Done uploading segments... Aug 7 21:20:53.942206917 (11h41m15.686263767s) [upload=12106]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.000728201s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [.......................................................XX...] duration:
319.244073ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 21:20:48 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 319.273404ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...X.......X................................................] duration:
874.088573ms
FAIL (2025-08-07 21:20:47 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
68895 (10h54m20.003725754s) [upload=11347]: Done uploading segments...
Aug 7 20:33:58.259694049 (10h54m20.00375091s) [upload=11347]: Upload finished. err: <nil>
Aug 7 20:34:42.291300370 (10h55m4.035357231s) [upload=11357]: Starting upload
Aug 7 20:34:42.291392440 (10h55m4.035449315s) [upload=11357]: Done starting upload
Aug 7 20:34:42.291505939 (10h55m4.035562788s) [upload=11357]: Uploading segments...
Aug 7 20:34:42.291587798 (10h55m4.035644667s) [upload=11357]: Got next segment. Inline: true
Aug 7 20:34:42.291610428 (10h55m4.035667289s) [upload=11357]: Waiting for error group managing segments...
Aug 7 20:34:42.291660078 (10h55m4.035716927s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:34:42.291665407 (10h55m4.035722257s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:34:42.291668818 (10h55m4.035725679s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:34:42.445169337 (10h55m4.189226185s) [upload=11357]: Done uploading segments...
Aug 7 20:34:42.445187662 (10h55m4.189244508s) [upload=11357]: Upload finished. err: <nil>
Aug 7 20:35:56.993979713 (10h56m18.738036563s) [upload=11378]: Starting upload
Aug 7 20:35:56.994042734 (10h56m18.738099594s) [upload=11378]: Done starting upload
Aug 7 20:35:56.994133564 (10h56m18.738190418s) [upload=11378]: Uploading segments...
Aug 7 20:35:56.994188836 (10h56m18.738245705s) [upload=11378]: Got next segment. Inline: true
Aug 7 20:35:56.994194253 (10h56m18.738251113s) [upload=11378]: Waiting for error group managing segments...
Aug 7 20:35:56.994255059 (10h56m18.738311921s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:35:56.994261721 (10h56m18.738318586s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:35:56.994266228 (10h56m18.73832309s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:35:57.196915243 (10h56m18.940972094s) [upload=11378]: Done uploading segments...
Aug 7 20:35:57.196933580 (10h56m18.940990431s) [upload=11378]: Upload finished. err: <nil>
Aug 7 20:36:48.383926585 (10h57m10.127983431s) [upload=11392]: Starting upload
Aug 7 20:36:48.383977279 (10h57m10.128034139s) [upload=11392]: Done starting upload
Aug 7 20:36:48.384080928 (10h57m10.128137781s) [upload=11392]: Uploading segments...
Aug 7 20:36:48.384124913 (10h57m10.128181781s) [upload=11392]: Got next segment. Inline: true
Aug 7 20:36:48.384130188 (10h57m10.128187037s) [upload=11392]: Waiting for error group managing segments...
Aug 7 20:36:48.384183298 (10h57m10.128240148s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:36:48.384188890 (10h57m10.128245741s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:36:48.384192353 (10h57m10.128249203s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:36:48.597716681 (10h57m10.341773532s) [upload=11392]: Done uploading segments...
Aug 7 20:36:48.597735626 (10h57m10.341792476s) [upload=11392]: Upload finished. err: <nil>
Aug 7 20:37:59.886756865 (10h58m21.630813711s) [upload=11411]: Starting upload
Aug 7 20:37:59.886810662 (10h58m21.630867522s) [upload=11411]: Done starting upload
Aug 7 20:37:59.886870714 (10h58m21.630927561s) [upload=11411]: Uploading segments...
Aug 7 20:37:59.886906763 (10h58m21.630963634s) [upload=11411]: Got next segment. Inline: true
Aug 7 20:37:59.886911413 (10h58m21.630968259s) [upload=11411]: Waiting for error group managing segments...
Aug 7 20:37:59.886948286 (10h58m21.631005132s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:37:59.886952923 (10h58m21.631009769s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:37:59.886957250 (10h58m21.631014096s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:38:00.073123067 (10h58m21.817179918s) [upload=11411]: Done uploading segments...
Aug 7 20:38:00.073142248 (10h58m21.817199097s) [upload=11411]: Upload finished. err: <nil>
Aug 7 20:39:06.192304563 (10h59m27.936361423s) [upload=11429]: Starting upload
Aug 7 20:39:06.192379945 (10h59m27.936436804s) [upload=11429]: Done starting upload
Aug 7 20:39:06.192450925 (10h59m27.936507772s) [upload=11429]: Uploading segments...
Aug 7 20:39:06.192493866 (10h59m27.936550726s) [upload=11429]: Got next segment. Inline: true
Aug 7 20:39:06.192499400 (10h59m27.93655626s) [upload=11429]: Waiting for error group managing segments...
Aug 7 20:39:06.192549058 (10h59m27.936605918s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:39:06.192553658 (10h59m27.936610518s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:39:06.192556283 (10h59m27.936613143s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:39:06.410732058 (10h59m28.154788918s) [upload=11429]: Done uploading segments...
Aug 7 20:39:06.410751072 (10h59m28.154807921s) [upload=11429]: Upload finished. err: <nil>
Aug 7 20:39:38.189437550 (10h59m59.9334944s) [upload=11438]: Starting upload
Aug 7 20:39:38.189494699 (10h59m59.933551565s) [upload=11438]: Done starting upload
Aug 7 20:39:38.189570885 (10h59m59.933627735s) [upload=11438]: Uploading segments...
Aug 7 20:39:38.189613828 (10h59m59.933670698s) [upload=11438]: Got next segment. Inline: true
Aug 7 20:39:38.189618898 (10h59m59.933675758s) [upload=11438]: Waiting for error group managing segments...
Aug 7 20:39:38.189671547 (10h59m59.933728396s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:39:38.189676923 (10h59m59.933733774s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:39:38.189680363 (10h59m59.933737214s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:39:38.339370287 (11h0m0.083427138s) [upload=11438]: Done uploading segments...
Aug 7 20:39:38.339394965 (11h0m0.083451822s) [upload=11438]: Upload finished. err: <nil>
Aug 7 20:40:13.049368895 (11h0m34.793425745s) [upload=11448]: Starting upload
Aug 7 20:40:13.049412890 (11h0m34.793469743s) [upload=11448]: Done starting upload
Aug 7 20:40:13.049490173 (11h0m34.793547027s) [upload=11448]: Uploading segments...
Aug 7 20:40:13.049531553 (11h0m34.793588423s) [upload=11448]: Got next segment. Inline: true
Aug 7 20:40:13.049536305 (11h0m34.79359316s) [upload=11448]: Waiting for error group managing segments...
Aug 7 20:40:13.049606089 (11h0m34.793662937s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:40:13.049611863 (11h0m34.793668715s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:40:13.049615487 (11h0m34.793672338s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:40:13.251485869 (11h0m34.995542721s) [upload=11448]: Done uploading segments...
Aug 7 20:40:13.251504252 (11h0m34.995561102s) [upload=11448]: Upload finished. err: <nil>
Aug 7 20:41:01.300323709 (11h1m23.044380561s) [upload=11462]: Starting upload
Aug 7 20:41:01.300386685 (11h1m23.044443554s) [upload=11462]: Done starting upload
Aug 7 20:41:01.300471424 (11h1m23.044528277s) [upload=11462]: Uploading segments...
Aug 7 20:41:01.300528867 (11h1m23.044585736s) [upload=11462]: Got next segment. Inline: true
Aug 7 20:41:01.300534426 (11h1m23.044591286s) [upload=11462]: Waiting for error group managing segments...
Aug 7 20:41:01.300592418 (11h1m23.044649268s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:01.300598189 (11h1m23.04465505s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:01.300601616 (11h1m23.044658476s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:01.448083138 (11h1m23.192139989s) [upload=11462]: Done uploading segments...
Aug 7 20:41:01.448102791 (11h1m23.192159644s) [upload=11462]: Upload finished. err: <nil>
Aug 7 20:41:25.522931110 (11h1m47.266987978s) [upload=11468]: Starting upload
Aug 7 20:41:25.523039051 (11h1m47.267095924s) [upload=11468]: Done starting upload
Aug 7 20:41:25.523181759 (11h1m47.267238621s) [upload=11468]: Uploading segments...
Aug 7 20:41:25.523403073 (11h1m47.267459944s) [upload=11468]: Got next segment. Inline: true
Aug 7 20:41:25.523413059 (11h1m47.26746993s) [upload=11468]: Waiting for error group managing segments...
Aug 7 20:41:25.523492519 (11h1m47.267549389s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:25.523498700 (11h1m47.267555566s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:25.523502761 (11h1m47.267559629s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:25.840530122 (11h1m47.584586983s) [upload=11468]: Done uploading segments...
Aug 7 20:41:25.840544406 (11h1m47.584601253s) [upload=11468]: Upload finished. err: <nil>
Aug 7 20:41:49.037046771 (11h2m10.781103621s) [upload=11474]: Starting upload
Aug 7 20:41:49.037108376 (11h2m10.781165228s) [upload=11474]: Done starting upload
Aug 7 20:41:49.037181045 (11h2m10.781237895s) [upload=11474]: Uploading segments...
Aug 7 20:41:49.037238734 (11h2m10.781295606s) [upload=11474]: Got next segment. Inline: true
Aug 7 20:41:49.037244163 (11h2m10.781301028s) [upload=11474]: Waiting for error group managing segments...
Aug 7 20:41:49.037309765 (11h2m10.781366628s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:49.037314702 (11h2m10.781371562s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:49.037317447 (11h2m10.781374307s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:49.205260584 (11h2m10.949317434s) [upload=11474]: Done uploading segments...
Aug 7 20:41:49.205278299 (11h2m10.94933515s) [upload=11474]: Upload finished. err: <nil>
Duration: 874.140857ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration:
30.000954788s
FAIL (2025-08-07 21:20:35 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 30.000969315s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............................X......................X......] duration:
2.76649054s
FAIL (2025-08-07 21:20:31 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
oad finished. err: <nil>
Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload
Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload
Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments...
Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false
Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment...
Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment.
Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment
Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments...
Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces.
Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments...
Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil>
Duration: 2.766529886s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration:
30.002458181s
FAIL (2025-08-07 21:20:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.002478144s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.........................................................XXX] duration:
30.00085591s
FAIL (2025-08-07 21:20:15 UTC)
orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.000875159s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
10 out of 60 is failed (threshold=3) [................................................X..XXXXXXXXX] duration:
30.001455774s
FAIL (2025-08-07 21:20:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.00147516s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration:
30.001161004s
FAIL (2025-08-07 21:20:05 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 30.001189822s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....X.......X...............................................] duration:
905.153429ms
FAIL (2025-08-07 21:20:04 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
68895 (10h54m20.003725754s) [upload=11347]: Done uploading segments...
Aug 7 20:33:58.259694049 (10h54m20.00375091s) [upload=11347]: Upload finished. err: <nil>
Aug 7 20:34:42.291300370 (10h55m4.035357231s) [upload=11357]: Starting upload
Aug 7 20:34:42.291392440 (10h55m4.035449315s) [upload=11357]: Done starting upload
Aug 7 20:34:42.291505939 (10h55m4.035562788s) [upload=11357]: Uploading segments...
Aug 7 20:34:42.291587798 (10h55m4.035644667s) [upload=11357]: Got next segment. Inline: true
Aug 7 20:34:42.291610428 (10h55m4.035667289s) [upload=11357]: Waiting for error group managing segments...
Aug 7 20:34:42.291660078 (10h55m4.035716927s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:34:42.291665407 (10h55m4.035722257s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:34:42.291668818 (10h55m4.035725679s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:34:42.445169337 (10h55m4.189226185s) [upload=11357]: Done uploading segments...
Aug 7 20:34:42.445187662 (10h55m4.189244508s) [upload=11357]: Upload finished. err: <nil>
Aug 7 20:35:56.993979713 (10h56m18.738036563s) [upload=11378]: Starting upload
Aug 7 20:35:56.994042734 (10h56m18.738099594s) [upload=11378]: Done starting upload
Aug 7 20:35:56.994133564 (10h56m18.738190418s) [upload=11378]: Uploading segments...
Aug 7 20:35:56.994188836 (10h56m18.738245705s) [upload=11378]: Got next segment. Inline: true
Aug 7 20:35:56.994194253 (10h56m18.738251113s) [upload=11378]: Waiting for error group managing segments...
Aug 7 20:35:56.994255059 (10h56m18.738311921s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:35:56.994261721 (10h56m18.738318586s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:35:56.994266228 (10h56m18.73832309s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:35:57.196915243 (10h56m18.940972094s) [upload=11378]: Done uploading segments...
Aug 7 20:35:57.196933580 (10h56m18.940990431s) [upload=11378]: Upload finished. err: <nil>
Aug 7 20:36:48.383926585 (10h57m10.127983431s) [upload=11392]: Starting upload
Aug 7 20:36:48.383977279 (10h57m10.128034139s) [upload=11392]: Done starting upload
Aug 7 20:36:48.384080928 (10h57m10.128137781s) [upload=11392]: Uploading segments...
Aug 7 20:36:48.384124913 (10h57m10.128181781s) [upload=11392]: Got next segment. Inline: true
Aug 7 20:36:48.384130188 (10h57m10.128187037s) [upload=11392]: Waiting for error group managing segments...
Aug 7 20:36:48.384183298 (10h57m10.128240148s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:36:48.384188890 (10h57m10.128245741s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:36:48.384192353 (10h57m10.128249203s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:36:48.597716681 (10h57m10.341773532s) [upload=11392]: Done uploading segments...
Aug 7 20:36:48.597735626 (10h57m10.341792476s) [upload=11392]: Upload finished. err: <nil>
Aug 7 20:37:59.886756865 (10h58m21.630813711s) [upload=11411]: Starting upload
Aug 7 20:37:59.886810662 (10h58m21.630867522s) [upload=11411]: Done starting upload
Aug 7 20:37:59.886870714 (10h58m21.630927561s) [upload=11411]: Uploading segments...
Aug 7 20:37:59.886906763 (10h58m21.630963634s) [upload=11411]: Got next segment. Inline: true
Aug 7 20:37:59.886911413 (10h58m21.630968259s) [upload=11411]: Waiting for error group managing segments...
Aug 7 20:37:59.886948286 (10h58m21.631005132s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:37:59.886952923 (10h58m21.631009769s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:37:59.886957250 (10h58m21.631014096s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:38:00.073123067 (10h58m21.817179918s) [upload=11411]: Done uploading segments...
Aug 7 20:38:00.073142248 (10h58m21.817199097s) [upload=11411]: Upload finished. err: <nil>
Aug 7 20:39:06.192304563 (10h59m27.936361423s) [upload=11429]: Starting upload
Aug 7 20:39:06.192379945 (10h59m27.936436804s) [upload=11429]: Done starting upload
Aug 7 20:39:06.192450925 (10h59m27.936507772s) [upload=11429]: Uploading segments...
Aug 7 20:39:06.192493866 (10h59m27.936550726s) [upload=11429]: Got next segment. Inline: true
Aug 7 20:39:06.192499400 (10h59m27.93655626s) [upload=11429]: Waiting for error group managing segments...
Aug 7 20:39:06.192549058 (10h59m27.936605918s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:39:06.192553658 (10h59m27.936610518s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:39:06.192556283 (10h59m27.936613143s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:39:06.410732058 (10h59m28.154788918s) [upload=11429]: Done uploading segments...
Aug 7 20:39:06.410751072 (10h59m28.154807921s) [upload=11429]: Upload finished. err: <nil>
Aug 7 20:39:38.189437550 (10h59m59.9334944s) [upload=11438]: Starting upload
Aug 7 20:39:38.189494699 (10h59m59.933551565s) [upload=11438]: Done starting upload
Aug 7 20:39:38.189570885 (10h59m59.933627735s) [upload=11438]: Uploading segments...
Aug 7 20:39:38.189613828 (10h59m59.933670698s) [upload=11438]: Got next segment. Inline: true
Aug 7 20:39:38.189618898 (10h59m59.933675758s) [upload=11438]: Waiting for error group managing segments...
Aug 7 20:39:38.189671547 (10h59m59.933728396s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:39:38.189676923 (10h59m59.933733774s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:39:38.189680363 (10h59m59.933737214s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:39:38.339370287 (11h0m0.083427138s) [upload=11438]: Done uploading segments...
Aug 7 20:39:38.339394965 (11h0m0.083451822s) [upload=11438]: Upload finished. err: <nil>
Aug 7 20:40:13.049368895 (11h0m34.793425745s) [upload=11448]: Starting upload
Aug 7 20:40:13.049412890 (11h0m34.793469743s) [upload=11448]: Done starting upload
Aug 7 20:40:13.049490173 (11h0m34.793547027s) [upload=11448]: Uploading segments...
Aug 7 20:40:13.049531553 (11h0m34.793588423s) [upload=11448]: Got next segment. Inline: true
Aug 7 20:40:13.049536305 (11h0m34.79359316s) [upload=11448]: Waiting for error group managing segments...
Aug 7 20:40:13.049606089 (11h0m34.793662937s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:40:13.049611863 (11h0m34.793668715s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:40:13.049615487 (11h0m34.793672338s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:40:13.251485869 (11h0m34.995542721s) [upload=11448]: Done uploading segments...
Aug 7 20:40:13.251504252 (11h0m34.995561102s) [upload=11448]: Upload finished. err: <nil>
Aug 7 20:41:01.300323709 (11h1m23.044380561s) [upload=11462]: Starting upload
Aug 7 20:41:01.300386685 (11h1m23.044443554s) [upload=11462]: Done starting upload
Aug 7 20:41:01.300471424 (11h1m23.044528277s) [upload=11462]: Uploading segments...
Aug 7 20:41:01.300528867 (11h1m23.044585736s) [upload=11462]: Got next segment. Inline: true
Aug 7 20:41:01.300534426 (11h1m23.044591286s) [upload=11462]: Waiting for error group managing segments...
Aug 7 20:41:01.300592418 (11h1m23.044649268s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:01.300598189 (11h1m23.04465505s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:01.300601616 (11h1m23.044658476s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:01.448083138 (11h1m23.192139989s) [upload=11462]: Done uploading segments...
Aug 7 20:41:01.448102791 (11h1m23.192159644s) [upload=11462]: Upload finished. err: <nil>
Aug 7 20:41:25.522931110 (11h1m47.266987978s) [upload=11468]: Starting upload
Aug 7 20:41:25.523039051 (11h1m47.267095924s) [upload=11468]: Done starting upload
Aug 7 20:41:25.523181759 (11h1m47.267238621s) [upload=11468]: Uploading segments...
Aug 7 20:41:25.523403073 (11h1m47.267459944s) [upload=11468]: Got next segment. Inline: true
Aug 7 20:41:25.523413059 (11h1m47.26746993s) [upload=11468]: Waiting for error group managing segments...
Aug 7 20:41:25.523492519 (11h1m47.267549389s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:25.523498700 (11h1m47.267555566s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:25.523502761 (11h1m47.267559629s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:25.840530122 (11h1m47.584586983s) [upload=11468]: Done uploading segments...
Aug 7 20:41:25.840544406 (11h1m47.584601253s) [upload=11468]: Upload finished. err: <nil>
Aug 7 20:41:49.037046771 (11h2m10.781103621s) [upload=11474]: Starting upload
Aug 7 20:41:49.037108376 (11h2m10.781165228s) [upload=11474]: Done starting upload
Aug 7 20:41:49.037181045 (11h2m10.781237895s) [upload=11474]: Uploading segments...
Aug 7 20:41:49.037238734 (11h2m10.781295606s) [upload=11474]: Got next segment. Inline: true
Aug 7 20:41:49.037244163 (11h2m10.781301028s) [upload=11474]: Waiting for error group managing segments...
Aug 7 20:41:49.037309765 (11h2m10.781366628s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:49.037314702 (11h2m10.781371562s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:49.037317447 (11h2m10.781374307s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:49.205260584 (11h2m10.949317434s) [upload=11474]: Done uploading segments...
Aug 7 20:41:49.205278299 (11h2m10.94933515s) [upload=11474]: Upload finished. err: <nil>
Duration: 905.19514ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
5 out of 60 is failed (threshold=3) [......................................................X.XXXX] duration:
1m0.001457295s
FAIL (2025-08-07 21:19:54 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 3243753s) [upload=11956]: Starting upload Aug 7 21:10:18.819260894 (11h30m40.563317754s) [upload=11956]: Done starting upload Aug 7 21:10:18.819487043 (11h30m40.563543905s) [upload=11956]: Uploading segments... Aug 7 21:10:18.819584609 (11h30m40.563641481s) [upload=11956]: Got next segment. Inline: true Aug 7 21:10:18.819614709 (11h30m40.563671559s) [upload=11956]: Waiting for error group managing segments... Aug 7 21:10:18.819668987 (11h30m40.563725851s) [upload=11956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:10:18.819689667 (11h30m40.563746534s) [upload=11956]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:10:18.819694129 (11h30m40.563750997s) [upload=11956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:10:18.958211137 (11h30m40.702268007s) [upload=11956]: Done uploading segments... Aug 7 21:10:18.958235626 (11h30m40.702292473s) [upload=11956]: Upload finished. err: <nil> Aug 7 21:11:16.699978190 (11h31m38.444035039s) [upload=11969]: Starting upload Aug 7 21:11:16.700090349 (11h31m38.44414722s) [upload=11969]: Done starting upload Aug 7 21:11:16.700171865 (11h31m38.444228712s) [upload=11969]: Uploading segments... Aug 7 21:11:16.700229408 (11h31m38.444286283s) [upload=11969]: Got next segment. Inline: true Aug 7 21:11:16.700236785 (11h31m38.444293651s) [upload=11969]: Waiting for error group managing segments... Aug 7 21:11:16.700303583 (11h31m38.444360444s) [upload=11969]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:11:16.700311256 (11h31m38.444368121s) [upload=11969]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:16.700316036 (11h31m38.444372904s) [upload=11969]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:11:16.845168013 (11h31m38.589224864s) [upload=11969]: Done uploading segments... Aug 7 21:11:16.845187939 (11h31m38.5892448s) [upload=11969]: Upload finished. err: <nil> Aug 7 21:12:04.147595666 (11h32m25.891652543s) [upload=11983]: Starting upload Aug 7 21:12:04.147723257 (11h32m25.891780138s) [upload=11983]: Done starting upload Aug 7 21:12:04.147862477 (11h32m25.891919358s) [upload=11983]: Uploading segments... Aug 7 21:12:04.147924428 (11h32m25.891981303s) [upload=11983]: Got next segment. Inline: true Aug 7 21:12:04.147931028 (11h32m25.89198791s) [upload=11983]: Waiting for error group managing segments... Aug 7 21:12:04.147990094 (11h32m25.892046969s) [upload=11983]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:12:04.147996570 (11h32m25.892053447s) [upload=11983]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:04.148003467 (11h32m25.892060346s) [upload=11983]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:12:04.368229522 (11h32m26.112286373s) [upload=11983]: Done uploading segments... Aug 7 21:12:04.368249120 (11h32m26.11230597s) [upload=11983]: Upload finished. err: <nil> Aug 7 21:12:47.792015773 (11h33m9.536072646s) [upload=11992]: Starting upload Aug 7 21:12:47.792127865 (11h33m9.536184732s) [upload=11992]: Done starting upload Aug 7 21:12:47.792248444 (11h33m9.536305307s) [upload=11992]: Uploading segments... Aug 7 21:12:47.792351067 (11h33m9.536407934s) [upload=11992]: Got next segment. Inline: true Aug 7 21:12:47.792359617 (11h33m9.536416486s) [upload=11992]: Waiting for error group managing segments... Aug 7 21:12:47.792414618 (11h33m9.536471489s) [upload=11992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:12:47.792420367 (11h33m9.536477238s) [upload=11992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:47.792423893 (11h33m9.536480763s) [upload=11992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:12:47.975624008 (11h33m9.71968088s) [upload=11992]: Done uploading segments... Aug 7 21:12:47.975648860 (11h33m9.71970573s) [upload=11992]: Upload finished. err: <nil> Aug 7 21:14:01.226859239 (11h34m22.970916111s) [upload=12013]: Starting upload Aug 7 21:14:01.227003646 (11h34m22.971060518s) [upload=12013]: Done starting upload Aug 7 21:14:01.227141194 (11h34m22.971198047s) [upload=12013]: Uploading segments... Aug 7 21:14:01.227219219 (11h34m22.971276084s) [upload=12013]: Got next segment. Inline: true Aug 7 21:14:01.227227170 (11h34m22.971284041s) [upload=12013]: Waiting for error group managing segments... Aug 7 21:14:01.227295298 (11h34m22.971352162s) [upload=12013]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:14:01.227301580 (11h34m22.971358443s) [upload=12013]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:01.227305300 (11h34m22.971362163s) [upload=12013]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:15:00.953618154 (11h35m22.697675004s) [upload=12013]: Done uploading segments... Aug 7 21:15:00.953640566 (11h35m22.697697426s) [upload=12013]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:01.394116473 (11h35m23.138173339s) [upload=12030]: Starting upload Aug 7 21:15:01.394242598 (11h35m23.138299459s) [upload=12030]: Done starting upload Aug 7 21:15:01.394373467 (11h35m23.138430318s) [upload=12030]: Uploading segments... Aug 7 21:15:01.394459055 (11h35m23.138515928s) [upload=12030]: Got next segment. Inline: true Aug 7 21:15:01.394467128 (11h35m23.138523996s) [upload=12030]: Waiting for error group managing segments... Aug 7 21:15:01.394548041 (11h35m23.138604915s) [upload=12030]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:15:01.394554816 (11h35m23.13861169s) [upload=12030]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:01.394559518 (11h35m23.138616392s) [upload=12030]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:15:03.584668408 (11h35m25.328725286s) [upload=12030]: Done uploading segments... Aug 7 21:15:03.584713689 (11h35m25.32877056s) [upload=12030]: Upload finished. err: <nil> Aug 7 21:15:54.636790163 (11h36m16.380847013s) [upload=12044]: Starting upload Aug 7 21:15:54.636877707 (11h36m16.380934638s) [upload=12044]: Done starting upload Aug 7 21:15:54.636936350 (11h36m16.3809932s) [upload=12044]: Uploading segments... Aug 7 21:15:54.636993824 (11h36m16.381050685s) [upload=12044]: Got next segment. Inline: true Aug 7 21:15:54.636999384 (11h36m16.381056244s) [upload=12044]: Waiting for error group managing segments... Aug 7 21:15:54.637044877 (11h36m16.381101727s) [upload=12044]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:15:54.637050662 (11h36m16.381107522s) [upload=12044]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:54.637053905 (11h36m16.381110766s) [upload=12044]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:54.359175610 (11h37m16.103232476s) [upload=12044]: Done uploading segments... Aug 7 21:16:54.359203156 (11h37m16.103260018s) [upload=12044]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:54.796260076 (11h37m16.54031694s) [upload=12060]: Starting upload Aug 7 21:16:54.796405798 (11h37m16.540462658s) [upload=12060]: Done starting upload Aug 7 21:16:54.796514557 (11h37m16.540571407s) [upload=12060]: Uploading segments... Aug 7 21:16:54.796581012 (11h37m16.540637875s) [upload=12060]: Got next segment. Inline: true Aug 7 21:16:54.796586298 (11h37m16.540643158s) [upload=12060]: Waiting for error group managing segments... Aug 7 21:16:54.796657533 (11h37m16.540714383s) [upload=12060]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:54.796663030 (11h37m16.540719894s) [upload=12060]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:54.796669424 (11h37m16.540726289s) [upload=12060]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:54.519327140 (11h38m16.263384004s) [upload=12060]: Done uploading segments... Aug 7 21:17:54.519368022 (11h38m16.263424882s) [upload=12060]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:54.944498267 (11h38m16.688555136s) [upload=12074]: Starting upload Aug 7 21:17:54.944617406 (11h38m16.688674267s) [upload=12074]: Done starting upload Aug 7 21:17:54.944695430 (11h38m16.688752285s) [upload=12074]: Uploading segments... Aug 7 21:17:54.944771865 (11h38m16.688828736s) [upload=12074]: Got next segment. Inline: true Aug 7 21:17:54.944781252 (11h38m16.688838123s) [upload=12074]: Waiting for error group managing segments... Aug 7 21:17:54.944841042 (11h38m16.688897916s) [upload=12074]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:54.944848212 (11h38m16.688905079s) [upload=12074]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:54.944851963 (11h38m16.688908827s) [upload=12074]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:54.673864467 (11h39m16.417921315s) [upload=12074]: Done uploading segments... Aug 7 21:18:54.673885859 (11h39m16.417942708s) [upload=12074]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:55.119912759 (11h39m16.863969619s) [upload=12087]: Starting upload Aug 7 21:18:55.120013170 (11h39m16.864070018s) [upload=12087]: Done starting upload Aug 7 21:18:55.120094735 (11h39m16.864151596s) [upload=12087]: Uploading segments... Aug 7 21:18:55.120142836 (11h39m16.864199705s) [upload=12087]: Got next segment. Inline: true Aug 7 21:18:55.120147776 (11h39m16.864204637s) [upload=12087]: Waiting for error group managing segments... Aug 7 21:18:55.120201712 (11h39m16.864258573s) [upload=12087]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:18:55.120207674 (11h39m16.864264534s) [upload=12087]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:55.120211321 (11h39m16.864268183s) [upload=12087]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:19:54.844368358 (11h40m16.588425206s) [upload=12087]: Done uploading segments... Aug 7 21:19:54.844395070 (11h40m16.588451926s) [upload=12087]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m0.00147519s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [........................................................XX..] duration:
387.376826ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 21:19:54 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 387.416657ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
10 out of 60 is failed (threshold=3) [.................................................XX.XXXXXXXX] duration:
30.001242463s
FAIL (2025-08-07 21:19:53 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== starting upload Aug 7 21:13:48.015414624 (11h34m9.759471489s) [upload=12011]: Uploading segments... Aug 7 21:13:48.015556587 (11h34m9.75961346s) [upload=12011]: Got next segment. Inline: true Aug 7 21:13:48.015566646 (11h34m9.759623517s) [upload=12011]: Waiting for error group managing segments... Aug 7 21:13:48.015627984 (11h34m9.759684854s) [upload=12011]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:13:48.015635040 (11h34m9.75969191s) [upload=12011]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:48.015638442 (11h34m9.759695306s) [upload=12011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:14:17.742337869 (11h34m39.486394729s) [upload=12011]: Done uploading segments... Aug 7 21:14:17.742364963 (11h34m39.486421823s) [upload=12011]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:14:18.164815368 (11h34m39.908872221s) [upload=12020]: Starting upload Aug 7 21:14:18.164900260 (11h34m39.908957119s) [upload=12020]: Done starting upload Aug 7 21:14:18.164985120 (11h34m39.909041981s) [upload=12020]: Uploading segments... Aug 7 21:14:18.165076859 (11h34m39.909133729s) [upload=12020]: Got next segment. Inline: true Aug 7 21:14:18.165084880 (11h34m39.909141745s) [upload=12020]: Waiting for error group managing segments... Aug 7 21:14:18.165136947 (11h34m39.909193807s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:14:18.165142336 (11h34m39.909199196s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:18.165145871 (11h34m39.909202731s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:14:18.323816200 (11h34m40.067873054s) [upload=12020]: Done uploading segments... Aug 7 21:14:18.323839479 (11h34m40.067896328s) [upload=12020]: Upload finished. err: <nil> Aug 7 21:15:30.409753622 (11h35m52.153810483s) [upload=12036]: Starting upload Aug 7 21:15:30.409818667 (11h35m52.153875531s) [upload=12036]: Done starting upload Aug 7 21:15:30.409895806 (11h35m52.153952669s) [upload=12036]: Uploading segments... Aug 7 21:15:30.409994454 (11h35m52.154051324s) [upload=12036]: Got next segment. Inline: true Aug 7 21:15:30.410001467 (11h35m52.154058337s) [upload=12036]: Waiting for error group managing segments... Aug 7 21:15:30.410069055 (11h35m52.154125913s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:15:30.410077097 (11h35m52.154133959s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:30.410082368 (11h35m52.154139256s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:00.139341290 (11h36m21.883398189s) [upload=12036]: Done uploading segments... Aug 7 21:16:00.139580695 (11h36m21.88363756s) [upload=12036]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:00.566884400 (11h36m22.310941251s) [upload=12047]: Starting upload Aug 7 21:16:00.566993382 (11h36m22.311050244s) [upload=12047]: Done starting upload Aug 7 21:16:00.567095627 (11h36m22.311152478s) [upload=12047]: Uploading segments... Aug 7 21:16:00.567233167 (11h36m22.311290039s) [upload=12047]: Got next segment. Inline: true Aug 7 21:16:00.567240534 (11h36m22.311297398s) [upload=12047]: Waiting for error group managing segments... Aug 7 21:16:00.567303473 (11h36m22.311360333s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:00.567314947 (11h36m22.311371807s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:00.567322295 (11h36m22.311379156s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:30.298703441 (11h36m52.0427603s) [upload=12047]: Done uploading segments... Aug 7 21:16:30.298732421 (11h36m52.04278928s) [upload=12047]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:41.168277425 (11h37m2.912334286s) [upload=12055]: Starting upload Aug 7 21:16:41.168334314 (11h37m2.912391183s) [upload=12055]: Done starting upload Aug 7 21:16:41.168407611 (11h37m2.912464477s) [upload=12055]: Uploading segments... Aug 7 21:16:41.168464727 (11h37m2.912521594s) [upload=12055]: Got next segment. Inline: true Aug 7 21:16:41.168470534 (11h37m2.912527404s) [upload=12055]: Waiting for error group managing segments... Aug 7 21:16:41.168520756 (11h37m2.912577617s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:41.168526439 (11h37m2.912583303s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:41.168529826 (11h37m2.91258669s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:10.898488630 (11h37m32.642545483s) [upload=12055]: Done uploading segments... Aug 7 21:17:10.898512884 (11h37m32.642569743s) [upload=12055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:11.314708421 (11h37m33.058765271s) [upload=12064]: Starting upload Aug 7 21:17:11.314772571 (11h37m33.058829423s) [upload=12064]: Done starting upload Aug 7 21:17:11.314846098 (11h37m33.05890295s) [upload=12064]: Uploading segments... Aug 7 21:17:11.314935220 (11h37m33.058992089s) [upload=12064]: Got next segment. Inline: true Aug 7 21:17:11.314940559 (11h37m33.058997419s) [upload=12064]: Waiting for error group managing segments... Aug 7 21:17:11.314984274 (11h37m33.059041125s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:11.314989343 (11h37m33.059046203s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:11.314992536 (11h37m33.059049396s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:41.043046082 (11h38m2.787102951s) [upload=12064]: Done uploading segments... Aug 7 21:17:41.043076101 (11h38m2.787132966s) [upload=12064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:41.498964594 (11h38m3.243021445s) [upload=12072]: Starting upload Aug 7 21:17:41.499038639 (11h38m3.243095509s) [upload=12072]: Done starting upload Aug 7 21:17:41.499118466 (11h38m3.243175318s) [upload=12072]: Uploading segments... Aug 7 21:17:41.499176424 (11h38m3.243233293s) [upload=12072]: Got next segment. Inline: true Aug 7 21:17:41.499181917 (11h38m3.243238777s) [upload=12072]: Waiting for error group managing segments... Aug 7 21:17:41.499236606 (11h38m3.243293472s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:41.499242465 (11h38m3.24329933s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:41.499245854 (11h38m3.243302721s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:11.231025141 (11h38m32.97508201s) [upload=12072]: Done uploading segments... Aug 7 21:18:11.231066796 (11h38m32.975123655s) [upload=12072]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:11.675991066 (11h38m33.420047932s) [upload=12078]: Starting upload Aug 7 21:18:11.676081621 (11h38m33.42013848s) [upload=12078]: Done starting upload Aug 7 21:18:11.676172293 (11h38m33.420229154s) [upload=12078]: Uploading segments... Aug 7 21:18:11.676235479 (11h38m33.420292347s) [upload=12078]: Got next segment. Inline: true Aug 7 21:18:11.676241644 (11h38m33.420298508s) [upload=12078]: Waiting for error group managing segments... Aug 7 21:18:11.676304521 (11h38m33.420361386s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:18:11.676310489 (11h38m33.420367352s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:11.676314037 (11h38m33.420370906s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:41.388891420 (11h39m3.132948271s) [upload=12078]: Done uploading segments... Aug 7 21:18:41.388921897 (11h39m3.132978749s) [upload=12078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:53.498741452 (11h39m15.242798302s) [upload=12086]: Starting upload Aug 7 21:18:53.498798950 (11h39m15.242855819s) [upload=12086]: Done starting upload Aug 7 21:18:53.498860950 (11h39m15.242917802s) [upload=12086]: Uploading segments... Aug 7 21:18:53.498901325 (11h39m15.242958195s) [upload=12086]: Got next segment. Inline: true Aug 7 21:18:53.498908406 (11h39m15.242965267s) [upload=12086]: Waiting for error group managing segments... Aug 7 21:18:53.498951359 (11h39m15.24300821s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:18:53.498956808 (11h39m15.243013668s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:53.498960205 (11h39m15.243017065s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:19:23.226932815 (11h39m44.970989679s) [upload=12086]: Done uploading segments... Aug 7 21:19:23.226954988 (11h39m44.971011848s) [upload=12086]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:19:23.633524671 (11h39m45.37758152s) [upload=12093]: Starting upload Aug 7 21:19:23.633599963 (11h39m45.377656832s) [upload=12093]: Done starting upload Aug 7 21:19:23.633677463 (11h39m45.377734313s) [upload=12093]: Uploading segments... Aug 7 21:19:23.633732206 (11h39m45.377789075s) [upload=12093]: Got next segment. Inline: true Aug 7 21:19:23.633737928 (11h39m45.377794778s) [upload=12093]: Waiting for error group managing segments... Aug 7 21:19:23.633790184 (11h39m45.377847035s) [upload=12093]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:19:23.633795686 (11h39m45.377852546s) [upload=12093]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:19:23.633801264 (11h39m45.377858124s) [upload=12093]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:19:53.363781250 (11h40m15.107838111s) [upload=12093]: Done uploading segments... Aug 7 21:19:53.363801573 (11h40m15.107858434s) [upload=12093]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.001262877s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration:
1m0.001438309s
FAIL (2025-08-07 21:19:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001461892s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
4m5.745029551s
FAIL (2025-08-07 21:19:45 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4m5.745066667s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
9 out of 60 is failed (threshold=3) [.................................................X..XXXXXXXX] duration:
30.000544433s
FAIL (2025-08-07 21:19:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.00057544s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration:
30.001043381s
FAIL (2025-08-07 21:19:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001085326s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
9 out of 60 is failed (threshold=3) [..................................................XX.XXXXXXX] duration:
30.000699739s
FAIL (2025-08-07 21:19:23 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== one starting upload Aug 7 21:12:53.894212135 (11h33m15.638268984s) [upload=11996]: Uploading segments... Aug 7 21:12:53.894264850 (11h33m15.638321716s) [upload=11996]: Got next segment. Inline: true Aug 7 21:12:53.894270475 (11h33m15.638327327s) [upload=11996]: Waiting for error group managing segments... Aug 7 21:12:53.894325009 (11h33m15.638381859s) [upload=11996]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:12:53.894330697 (11h33m15.638387548s) [upload=11996]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:53.894334066 (11h33m15.638390917s) [upload=11996]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:13:23.625409415 (11h33m45.36946628s) [upload=11996]: Done uploading segments... Aug 7 21:13:23.625434285 (11h33m45.369491153s) [upload=11996]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:13:48.015197539 (11h34m9.7592544s) [upload=12011]: Starting upload Aug 7 21:13:48.015280384 (11h34m9.759337244s) [upload=12011]: Done starting upload Aug 7 21:13:48.015414624 (11h34m9.759471489s) [upload=12011]: Uploading segments... Aug 7 21:13:48.015556587 (11h34m9.75961346s) [upload=12011]: Got next segment. Inline: true Aug 7 21:13:48.015566646 (11h34m9.759623517s) [upload=12011]: Waiting for error group managing segments... Aug 7 21:13:48.015627984 (11h34m9.759684854s) [upload=12011]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:13:48.015635040 (11h34m9.75969191s) [upload=12011]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:48.015638442 (11h34m9.759695306s) [upload=12011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:14:17.742337869 (11h34m39.486394729s) [upload=12011]: Done uploading segments... Aug 7 21:14:17.742364963 (11h34m39.486421823s) [upload=12011]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:14:18.164815368 (11h34m39.908872221s) [upload=12020]: Starting upload Aug 7 21:14:18.164900260 (11h34m39.908957119s) [upload=12020]: Done starting upload Aug 7 21:14:18.164985120 (11h34m39.909041981s) [upload=12020]: Uploading segments... Aug 7 21:14:18.165076859 (11h34m39.909133729s) [upload=12020]: Got next segment. Inline: true Aug 7 21:14:18.165084880 (11h34m39.909141745s) [upload=12020]: Waiting for error group managing segments... Aug 7 21:14:18.165136947 (11h34m39.909193807s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:14:18.165142336 (11h34m39.909199196s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:18.165145871 (11h34m39.909202731s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:14:18.323816200 (11h34m40.067873054s) [upload=12020]: Done uploading segments... Aug 7 21:14:18.323839479 (11h34m40.067896328s) [upload=12020]: Upload finished. err: <nil> Aug 7 21:15:30.409753622 (11h35m52.153810483s) [upload=12036]: Starting upload Aug 7 21:15:30.409818667 (11h35m52.153875531s) [upload=12036]: Done starting upload Aug 7 21:15:30.409895806 (11h35m52.153952669s) [upload=12036]: Uploading segments... Aug 7 21:15:30.409994454 (11h35m52.154051324s) [upload=12036]: Got next segment. Inline: true Aug 7 21:15:30.410001467 (11h35m52.154058337s) [upload=12036]: Waiting for error group managing segments... Aug 7 21:15:30.410069055 (11h35m52.154125913s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:15:30.410077097 (11h35m52.154133959s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:30.410082368 (11h35m52.154139256s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:00.139341290 (11h36m21.883398189s) [upload=12036]: Done uploading segments... Aug 7 21:16:00.139580695 (11h36m21.88363756s) [upload=12036]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:00.566884400 (11h36m22.310941251s) [upload=12047]: Starting upload Aug 7 21:16:00.566993382 (11h36m22.311050244s) [upload=12047]: Done starting upload Aug 7 21:16:00.567095627 (11h36m22.311152478s) [upload=12047]: Uploading segments... Aug 7 21:16:00.567233167 (11h36m22.311290039s) [upload=12047]: Got next segment. Inline: true Aug 7 21:16:00.567240534 (11h36m22.311297398s) [upload=12047]: Waiting for error group managing segments... Aug 7 21:16:00.567303473 (11h36m22.311360333s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:00.567314947 (11h36m22.311371807s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:00.567322295 (11h36m22.311379156s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:30.298703441 (11h36m52.0427603s) [upload=12047]: Done uploading segments... Aug 7 21:16:30.298732421 (11h36m52.04278928s) [upload=12047]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:41.168277425 (11h37m2.912334286s) [upload=12055]: Starting upload Aug 7 21:16:41.168334314 (11h37m2.912391183s) [upload=12055]: Done starting upload Aug 7 21:16:41.168407611 (11h37m2.912464477s) [upload=12055]: Uploading segments... Aug 7 21:16:41.168464727 (11h37m2.912521594s) [upload=12055]: Got next segment. Inline: true Aug 7 21:16:41.168470534 (11h37m2.912527404s) [upload=12055]: Waiting for error group managing segments... Aug 7 21:16:41.168520756 (11h37m2.912577617s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:41.168526439 (11h37m2.912583303s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:41.168529826 (11h37m2.91258669s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:10.898488630 (11h37m32.642545483s) [upload=12055]: Done uploading segments... Aug 7 21:17:10.898512884 (11h37m32.642569743s) [upload=12055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:11.314708421 (11h37m33.058765271s) [upload=12064]: Starting upload Aug 7 21:17:11.314772571 (11h37m33.058829423s) [upload=12064]: Done starting upload Aug 7 21:17:11.314846098 (11h37m33.05890295s) [upload=12064]: Uploading segments... Aug 7 21:17:11.314935220 (11h37m33.058992089s) [upload=12064]: Got next segment. Inline: true Aug 7 21:17:11.314940559 (11h37m33.058997419s) [upload=12064]: Waiting for error group managing segments... Aug 7 21:17:11.314984274 (11h37m33.059041125s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:11.314989343 (11h37m33.059046203s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:11.314992536 (11h37m33.059049396s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:41.043046082 (11h38m2.787102951s) [upload=12064]: Done uploading segments... Aug 7 21:17:41.043076101 (11h38m2.787132966s) [upload=12064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:41.498964594 (11h38m3.243021445s) [upload=12072]: Starting upload Aug 7 21:17:41.499038639 (11h38m3.243095509s) [upload=12072]: Done starting upload Aug 7 21:17:41.499118466 (11h38m3.243175318s) [upload=12072]: Uploading segments... Aug 7 21:17:41.499176424 (11h38m3.243233293s) [upload=12072]: Got next segment. Inline: true Aug 7 21:17:41.499181917 (11h38m3.243238777s) [upload=12072]: Waiting for error group managing segments... Aug 7 21:17:41.499236606 (11h38m3.243293472s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:41.499242465 (11h38m3.24329933s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:41.499245854 (11h38m3.243302721s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:11.231025141 (11h38m32.97508201s) [upload=12072]: Done uploading segments... Aug 7 21:18:11.231066796 (11h38m32.975123655s) [upload=12072]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:11.675991066 (11h38m33.420047932s) [upload=12078]: Starting upload Aug 7 21:18:11.676081621 (11h38m33.42013848s) [upload=12078]: Done starting upload Aug 7 21:18:11.676172293 (11h38m33.420229154s) [upload=12078]: Uploading segments... Aug 7 21:18:11.676235479 (11h38m33.420292347s) [upload=12078]: Got next segment. Inline: true Aug 7 21:18:11.676241644 (11h38m33.420298508s) [upload=12078]: Waiting for error group managing segments... Aug 7 21:18:11.676304521 (11h38m33.420361386s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:18:11.676310489 (11h38m33.420367352s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:11.676314037 (11h38m33.420370906s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:41.388891420 (11h39m3.132948271s) [upload=12078]: Done uploading segments... Aug 7 21:18:41.388921897 (11h39m3.132978749s) [upload=12078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:53.498741452 (11h39m15.242798302s) [upload=12086]: Starting upload Aug 7 21:18:53.498798950 (11h39m15.242855819s) [upload=12086]: Done starting upload Aug 7 21:18:53.498860950 (11h39m15.242917802s) [upload=12086]: Uploading segments... Aug 7 21:18:53.498901325 (11h39m15.242958195s) [upload=12086]: Got next segment. Inline: true Aug 7 21:18:53.498908406 (11h39m15.242965267s) [upload=12086]: Waiting for error group managing segments... Aug 7 21:18:53.498951359 (11h39m15.24300821s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:18:53.498956808 (11h39m15.243013668s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:53.498960205 (11h39m15.243017065s) [upload=12086]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:19:23.226932815 (11h39m44.970989679s) [upload=12086]: Done uploading segments... Aug 7 21:19:23.226954988 (11h39m44.971011848s) [upload=12086]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.000724952s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............................X......................X.....] duration:
3.161932705s
FAIL (2025-08-07 21:19:18 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
oad finished. err: <nil>
Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload
Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload
Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments...
Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false
Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment...
Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment.
Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment
Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments...
Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces.
Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments...
Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil>
Duration: 3.161969281s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration:
30.000566481s
FAIL (2025-08-07 21:19:10 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: context deadline exceeded
======LOGS======
4.668165335s) [noise=true , address=45.91.116.8:4045 , piece_id=2NUYYQ3WGF2UO2ROSY3R6FN43YEYFEPHGAK6HHV4MXNZIAUCB6JA, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.936065869 (11h33m54.680122736s) [noise=true , address=77.90.31.12:4589 , piece_id=DQLSJRXEL4SG63R6DH6YPSKBRFCCDZ2M6VS5NX2X72LKQ4XMHH6A, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.943684467 (11h33m54.687741333s) [noise=true , address=45.91.116.13:4153 , piece_id=VCQLYREVBH3VTQVVQ4N6RJJKIA5RTPLROMUZKFH62A3P7B4GLFTQ, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.969180656 (11h33m54.713237508s) [noise=true , address=77.90.31.19:4843 , piece_id=3FPX5CXVNG7IFODGUVUYL3V6RECWAGN7SYDS2MXEUE6FECWWCHZA, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.986237351 (11h33m54.730294197s) [noise=true , address=45.91.116.11:4117 , piece_id=TH2ZUH4NO4YPRUNW4ABP2BLBXEVQB2KVEJEK3COYPQGMG3AISQBQ, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.986598715 (11h33m54.730655567s) [noise=true , address=77.90.31.13:4628 , piece_id=5L2DJMNTXMXQC4NNUW742I4RYY5U4JZCW66MUW4C7R6A3IJ46WHA, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.993716558 (11h33m54.73777342s) [noise=true , address=77.90.31.15:4699 , piece_id=4QFC6XUXFSCGNCVUQ6APWFP4VPOT2XEPBF6MLLNJPWJIAHWTNAQQ, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:33.004764885 (11h33m54.748821753s) [noise=true , address=45.91.116.12:4225 , piece_id=WBWYO5U3GRZX4MAS5TVYV4X75LM2IM4MYAALYWITSTJY3G6C6V3A, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:33.007513393 (11h33m54.75157026s) [noise=true , address=104.36.82.11:4114 , piece_id=CZPJM2V75OCGBKDT3VONHMK4UK7UV7W5CM7ALZV63Z5WA44R5KHA, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:33.007550343 (11h33m54.751607205s) [seg_pos={0 0}, upload=12005]: Segment reached optimal threshold of 13 pieces.
Aug 7 21:13:33.007818213 (11h33m54.75187508s) [noise=true , address=104.36.82.9:4006 , piece_id=DUDTTETXBUSMAJP7HWVVVU6JNB3NODJEYNKCWJWVIZYACCPF7KIQ, seg_pos={0 0}, upload=12005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: expected piece hash; context canceled; EOF
Aug 7 21:13:33.007957507 (11h33m54.752014368s) [noise=true , address=45.91.116.9:4263 , piece_id=U4WUKRSZCL757XKP25KUR6CQ7GNX2NQ544SBF7DEP4DLKDLQCT5A, seg_pos={0 0}, upload=12005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF
Aug 7 21:13:33.008072807 (11h33m54.752129674s) [noise=true , address=45.91.116.10:4082 , piece_id=OOR4QA3CXAA7G4AJRO2MJVKKAPWNMWXEG4MGPHFSSEJJG2PAP3MA, seg_pos={0 0}, upload=12005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF
Aug 7 21:13:33.008088785 (11h33m54.752145655s) [seg_pos={0 0}, upload=12005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 7 21:13:33.008239833 (11h33m54.752296703s) [upload=12005]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:13:33.008246480 (11h33m54.752303365s) [upload=12005]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:13:56.682120899 (11h34m18.426177749s) [upload=12005]: Done uploading segments...
Aug 7 21:13:56.682153262 (11h34m18.426210125s) [upload=12005]: Upload finished. err: metaclient: context deadline exceeded
Aug 7 21:14:08.981143212 (11h34m30.725200099s) [upload=12017]: Starting upload
Aug 7 21:14:08.981247915 (11h34m30.725304782s) [upload=12017]: Done starting upload
Aug 7 21:14:08.981457434 (11h34m30.725514297s) [upload=12017]: Uploading segments...
Aug 7 21:14:08.981983258 (11h34m30.726040126s) [upload=12017]: Got next segment. Inline: false
Aug 7 21:14:08.982063476 (11h34m30.726120349s) [upload=12017]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:14:08.982070288 (11h34m30.726127163s) [upload=12017]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:14:38.713635311 (11h35m0.457692174s) [upload=12017]: Next segment err: context deadline exceeded
Aug 7 21:14:38.713818148 (11h35m0.457874995s) [upload=12017]: Done uploading segments...
Aug 7 21:14:38.713847460 (11h35m0.457904307s) [upload=12017]: Upload finished. err: context deadline exceeded
Aug 7 21:14:39.126797864 (11h35m0.870854713s) [upload=12024]: Starting upload
Aug 7 21:14:39.126867479 (11h35m0.87092433s) [upload=12024]: Done starting upload
Aug 7 21:14:39.127134096 (11h35m0.871190958s) [upload=12024]: Uploading segments...
Aug 7 21:14:39.127206037 (11h35m0.871262906s) [upload=12024]: Got next segment. Inline: false
Aug 7 21:14:39.127247021 (11h35m0.871303875s) [upload=12024]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:14:39.127254726 (11h35m0.871311587s) [upload=12024]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:15:08.857744672 (11h35m30.601801542s) [upload=12024]: Next segment err: context deadline exceeded
Aug 7 21:15:08.857937199 (11h35m30.60199407s) [upload=12024]: Done uploading segments...
Aug 7 21:15:08.857962321 (11h35m30.602019169s) [upload=12024]: Upload finished. err: context deadline exceeded
Aug 7 21:15:20.485893561 (11h35m42.229950429s) [upload=12034]: Starting upload
Aug 7 21:15:20.486023409 (11h35m42.230080283s) [upload=12034]: Done starting upload
Aug 7 21:15:20.486134808 (11h35m42.230191661s) [upload=12034]: Uploading segments...
Aug 7 21:15:20.486431156 (11h35m42.230488009s) [upload=12034]: Got next segment. Inline: false
Aug 7 21:15:20.486483996 (11h35m42.230540863s) [upload=12034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:15:20.486491542 (11h35m42.230548409s) [upload=12034]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:15:50.215320677 (11h36m11.959377531s) [upload=12034]: Next segment err: context deadline exceeded
Aug 7 21:15:50.215551089 (11h36m11.95960795s) [upload=12034]: Done uploading segments...
Aug 7 21:15:50.215590354 (11h36m11.959647201s) [upload=12034]: Upload finished. err: context deadline exceeded
Aug 7 21:15:59.944633175 (11h36m21.688690025s) [upload=12045]: Starting upload
Aug 7 21:15:59.944712435 (11h36m21.6887693s) [upload=12045]: Done starting upload
Aug 7 21:15:59.945075324 (11h36m21.689132187s) [upload=12045]: Uploading segments...
Aug 7 21:15:59.945181160 (11h36m21.689238043s) [upload=12045]: Got next segment. Inline: false
Aug 7 21:15:59.945260803 (11h36m21.689317664s) [upload=12045]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:15:59.945267828 (11h36m21.68932469s) [upload=12045]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:16:29.659806113 (11h36m51.403862973s) [upload=12045]: Next segment err: context deadline exceeded
Aug 7 21:16:29.659892256 (11h36m51.403949103s) [upload=12045]: Done uploading segments...
Aug 7 21:16:29.659910327 (11h36m51.403967174s) [upload=12045]: Upload finished. err: context deadline exceeded
Aug 7 21:16:40.061254158 (11h37m1.80531101s) [upload=12054]: Starting upload
Aug 7 21:16:40.061330739 (11h37m1.805387599s) [upload=12054]: Done starting upload
Aug 7 21:16:40.061534219 (11h37m1.805591069s) [upload=12054]: Uploading segments...
Aug 7 21:16:40.061675268 (11h37m1.805732119s) [upload=12054]: Got next segment. Inline: false
Aug 7 21:16:40.061734809 (11h37m1.805791655s) [upload=12054]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:16:40.061740406 (11h37m1.805797266s) [upload=12054]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:17:09.783636482 (11h37m31.52769334s) [upload=12054]: Next segment err: context deadline exceeded
Aug 7 21:17:09.783809752 (11h37m31.527866609s) [upload=12054]: Done uploading segments...
Aug 7 21:17:09.783883524 (11h37m31.527940372s) [upload=12054]: Upload finished. err: context deadline exceeded
Aug 7 21:17:38.423613839 (11h38m0.16767071s) [upload=12070]: Starting upload
Aug 7 21:17:38.423678998 (11h38m0.167735863s) [upload=12070]: Done starting upload
Aug 7 21:17:38.423788045 (11h38m0.167844912s) [upload=12070]: Uploading segments...
Aug 7 21:17:38.423932717 (11h38m0.167989565s) [upload=12070]: Got next segment. Inline: false
Aug 7 21:17:38.424005392 (11h38m0.168062243s) [upload=12070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:17:38.424011139 (11h38m0.168068002s) [upload=12070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:18:08.145022440 (11h38m29.889079332s) [upload=12070]: Next segment err: context deadline exceeded
Aug 7 21:18:08.145219437 (11h38m29.889276305s) [upload=12070]: Done uploading segments...
Aug 7 21:18:08.145260561 (11h38m29.889317417s) [upload=12070]: Upload finished. err: context deadline exceeded
Aug 7 21:18:40.680413978 (11h39m2.424470847s) [upload=12083]: Starting upload
Aug 7 21:18:40.680537346 (11h39m2.424594209s) [upload=12083]: Done starting upload
Aug 7 21:18:40.680647310 (11h39m2.42470416s) [upload=12083]: Uploading segments...
Aug 7 21:18:40.680927321 (11h39m2.424984171s) [upload=12083]: Got next segment. Inline: false
Aug 7 21:18:40.680982611 (11h39m2.425039462s) [upload=12083]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:18:40.680988246 (11h39m2.425045096s) [upload=12083]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:19:10.409831637 (11h39m32.153888501s) [upload=12083]: Next segment err: context deadline exceeded
Aug 7 21:19:10.409958917 (11h39m32.154015763s) [upload=12083]: Done uploading segments...
Aug 7 21:19:10.410002320 (11h39m32.154059166s) [upload=12083]: Upload finished. err: context deadline exceeded
Duration: 30.000594825s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
8 out of 60 is failed (threshold=3) [..................................................X..XXXXXXX] duration:
30.001767661s
FAIL (2025-08-07 21:19:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001804928s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [.........................................................XX.] duration:
334.583791ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 21:19:03 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 334.606985ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
2m0.000645037s remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-07 21:19:02 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 2m0.000668874s
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration:
30.001574413s
FAIL (2025-08-07 21:19:00 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 30.001610282s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....X.......X..............................................] duration:
863.803372ms
FAIL (2025-08-07 21:18:57 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
68895 (10h54m20.003725754s) [upload=11347]: Done uploading segments...
Aug 7 20:33:58.259694049 (10h54m20.00375091s) [upload=11347]: Upload finished. err: <nil>
Aug 7 20:34:42.291300370 (10h55m4.035357231s) [upload=11357]: Starting upload
Aug 7 20:34:42.291392440 (10h55m4.035449315s) [upload=11357]: Done starting upload
Aug 7 20:34:42.291505939 (10h55m4.035562788s) [upload=11357]: Uploading segments...
Aug 7 20:34:42.291587798 (10h55m4.035644667s) [upload=11357]: Got next segment. Inline: true
Aug 7 20:34:42.291610428 (10h55m4.035667289s) [upload=11357]: Waiting for error group managing segments...
Aug 7 20:34:42.291660078 (10h55m4.035716927s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:34:42.291665407 (10h55m4.035722257s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:34:42.291668818 (10h55m4.035725679s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:34:42.445169337 (10h55m4.189226185s) [upload=11357]: Done uploading segments...
Aug 7 20:34:42.445187662 (10h55m4.189244508s) [upload=11357]: Upload finished. err: <nil>
Aug 7 20:35:56.993979713 (10h56m18.738036563s) [upload=11378]: Starting upload
Aug 7 20:35:56.994042734 (10h56m18.738099594s) [upload=11378]: Done starting upload
Aug 7 20:35:56.994133564 (10h56m18.738190418s) [upload=11378]: Uploading segments...
Aug 7 20:35:56.994188836 (10h56m18.738245705s) [upload=11378]: Got next segment. Inline: true
Aug 7 20:35:56.994194253 (10h56m18.738251113s) [upload=11378]: Waiting for error group managing segments...
Aug 7 20:35:56.994255059 (10h56m18.738311921s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:35:56.994261721 (10h56m18.738318586s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:35:56.994266228 (10h56m18.73832309s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:35:57.196915243 (10h56m18.940972094s) [upload=11378]: Done uploading segments...
Aug 7 20:35:57.196933580 (10h56m18.940990431s) [upload=11378]: Upload finished. err: <nil>
Aug 7 20:36:48.383926585 (10h57m10.127983431s) [upload=11392]: Starting upload
Aug 7 20:36:48.383977279 (10h57m10.128034139s) [upload=11392]: Done starting upload
Aug 7 20:36:48.384080928 (10h57m10.128137781s) [upload=11392]: Uploading segments...
Aug 7 20:36:48.384124913 (10h57m10.128181781s) [upload=11392]: Got next segment. Inline: true
Aug 7 20:36:48.384130188 (10h57m10.128187037s) [upload=11392]: Waiting for error group managing segments...
Aug 7 20:36:48.384183298 (10h57m10.128240148s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:36:48.384188890 (10h57m10.128245741s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:36:48.384192353 (10h57m10.128249203s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:36:48.597716681 (10h57m10.341773532s) [upload=11392]: Done uploading segments...
Aug 7 20:36:48.597735626 (10h57m10.341792476s) [upload=11392]: Upload finished. err: <nil>
Aug 7 20:37:59.886756865 (10h58m21.630813711s) [upload=11411]: Starting upload
Aug 7 20:37:59.886810662 (10h58m21.630867522s) [upload=11411]: Done starting upload
Aug 7 20:37:59.886870714 (10h58m21.630927561s) [upload=11411]: Uploading segments...
Aug 7 20:37:59.886906763 (10h58m21.630963634s) [upload=11411]: Got next segment. Inline: true
Aug 7 20:37:59.886911413 (10h58m21.630968259s) [upload=11411]: Waiting for error group managing segments...
Aug 7 20:37:59.886948286 (10h58m21.631005132s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:37:59.886952923 (10h58m21.631009769s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:37:59.886957250 (10h58m21.631014096s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:38:00.073123067 (10h58m21.817179918s) [upload=11411]: Done uploading segments...
Aug 7 20:38:00.073142248 (10h58m21.817199097s) [upload=11411]: Upload finished. err: <nil>
Aug 7 20:39:06.192304563 (10h59m27.936361423s) [upload=11429]: Starting upload
Aug 7 20:39:06.192379945 (10h59m27.936436804s) [upload=11429]: Done starting upload
Aug 7 20:39:06.192450925 (10h59m27.936507772s) [upload=11429]: Uploading segments...
Aug 7 20:39:06.192493866 (10h59m27.936550726s) [upload=11429]: Got next segment. Inline: true
Aug 7 20:39:06.192499400 (10h59m27.93655626s) [upload=11429]: Waiting for error group managing segments...
Aug 7 20:39:06.192549058 (10h59m27.936605918s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:39:06.192553658 (10h59m27.936610518s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:39:06.192556283 (10h59m27.936613143s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:39:06.410732058 (10h59m28.154788918s) [upload=11429]: Done uploading segments...
Aug 7 20:39:06.410751072 (10h59m28.154807921s) [upload=11429]: Upload finished. err: <nil>
Aug 7 20:39:38.189437550 (10h59m59.9334944s) [upload=11438]: Starting upload
Aug 7 20:39:38.189494699 (10h59m59.933551565s) [upload=11438]: Done starting upload
Aug 7 20:39:38.189570885 (10h59m59.933627735s) [upload=11438]: Uploading segments...
Aug 7 20:39:38.189613828 (10h59m59.933670698s) [upload=11438]: Got next segment. Inline: true
Aug 7 20:39:38.189618898 (10h59m59.933675758s) [upload=11438]: Waiting for error group managing segments...
Aug 7 20:39:38.189671547 (10h59m59.933728396s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:39:38.189676923 (10h59m59.933733774s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:39:38.189680363 (10h59m59.933737214s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:39:38.339370287 (11h0m0.083427138s) [upload=11438]: Done uploading segments...
Aug 7 20:39:38.339394965 (11h0m0.083451822s) [upload=11438]: Upload finished. err: <nil>
Aug 7 20:40:13.049368895 (11h0m34.793425745s) [upload=11448]: Starting upload
Aug 7 20:40:13.049412890 (11h0m34.793469743s) [upload=11448]: Done starting upload
Aug 7 20:40:13.049490173 (11h0m34.793547027s) [upload=11448]: Uploading segments...
Aug 7 20:40:13.049531553 (11h0m34.793588423s) [upload=11448]: Got next segment. Inline: true
Aug 7 20:40:13.049536305 (11h0m34.79359316s) [upload=11448]: Waiting for error group managing segments...
Aug 7 20:40:13.049606089 (11h0m34.793662937s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:40:13.049611863 (11h0m34.793668715s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:40:13.049615487 (11h0m34.793672338s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:40:13.251485869 (11h0m34.995542721s) [upload=11448]: Done uploading segments...
Aug 7 20:40:13.251504252 (11h0m34.995561102s) [upload=11448]: Upload finished. err: <nil>
Aug 7 20:41:01.300323709 (11h1m23.044380561s) [upload=11462]: Starting upload
Aug 7 20:41:01.300386685 (11h1m23.044443554s) [upload=11462]: Done starting upload
Aug 7 20:41:01.300471424 (11h1m23.044528277s) [upload=11462]: Uploading segments...
Aug 7 20:41:01.300528867 (11h1m23.044585736s) [upload=11462]: Got next segment. Inline: true
Aug 7 20:41:01.300534426 (11h1m23.044591286s) [upload=11462]: Waiting for error group managing segments...
Aug 7 20:41:01.300592418 (11h1m23.044649268s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:01.300598189 (11h1m23.04465505s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:01.300601616 (11h1m23.044658476s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:01.448083138 (11h1m23.192139989s) [upload=11462]: Done uploading segments...
Aug 7 20:41:01.448102791 (11h1m23.192159644s) [upload=11462]: Upload finished. err: <nil>
Aug 7 20:41:25.522931110 (11h1m47.266987978s) [upload=11468]: Starting upload
Aug 7 20:41:25.523039051 (11h1m47.267095924s) [upload=11468]: Done starting upload
Aug 7 20:41:25.523181759 (11h1m47.267238621s) [upload=11468]: Uploading segments...
Aug 7 20:41:25.523403073 (11h1m47.267459944s) [upload=11468]: Got next segment. Inline: true
Aug 7 20:41:25.523413059 (11h1m47.26746993s) [upload=11468]: Waiting for error group managing segments...
Aug 7 20:41:25.523492519 (11h1m47.267549389s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:25.523498700 (11h1m47.267555566s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:25.523502761 (11h1m47.267559629s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:25.840530122 (11h1m47.584586983s) [upload=11468]: Done uploading segments...
Aug 7 20:41:25.840544406 (11h1m47.584601253s) [upload=11468]: Upload finished. err: <nil>
Aug 7 20:41:49.037046771 (11h2m10.781103621s) [upload=11474]: Starting upload
Aug 7 20:41:49.037108376 (11h2m10.781165228s) [upload=11474]: Done starting upload
Aug 7 20:41:49.037181045 (11h2m10.781237895s) [upload=11474]: Uploading segments...
Aug 7 20:41:49.037238734 (11h2m10.781295606s) [upload=11474]: Got next segment. Inline: true
Aug 7 20:41:49.037244163 (11h2m10.781301028s) [upload=11474]: Waiting for error group managing segments...
Aug 7 20:41:49.037309765 (11h2m10.781366628s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:49.037314702 (11h2m10.781371562s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:49.037317447 (11h2m10.781374307s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:49.205260584 (11h2m10.949317434s) [upload=11474]: Done uploading segments...
Aug 7 20:41:49.205278299 (11h2m10.94933515s) [upload=11474]: Upload finished. err: <nil>
Duration: 863.843859ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
4 out of 60 is failed (threshold=3) [.......................................................X.XXX] duration:
1m0.000712175s
FAIL (2025-08-07 21:18:54 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 21:09:33.773325036 (11h29m55.517381899s) [upload=11943]: Starting upload Aug 7 21:09:33.773448919 (11h29m55.517505779s) [upload=11943]: Done starting upload Aug 7 21:09:33.773536590 (11h29m55.517593439s) [upload=11943]: Uploading segments... Aug 7 21:09:33.773595522 (11h29m55.517652391s) [upload=11943]: Got next segment. Inline: true Aug 7 21:09:33.773600689 (11h29m55.517657549s) [upload=11943]: Waiting for error group managing segments... Aug 7 21:09:33.773653328 (11h29m55.51771019s) [upload=11943]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:09:33.773658841 (11h29m55.517715702s) [upload=11943]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:09:33.773662446 (11h29m55.517719309s) [upload=11943]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:09:33.923851484 (11h29m55.667908344s) [upload=11943]: Done uploading segments... Aug 7 21:09:33.923871014 (11h29m55.667927875s) [upload=11943]: Upload finished. err: <nil> Aug 7 21:10:18.819186903 (11h30m40.563243753s) [upload=11956]: Starting upload Aug 7 21:10:18.819260894 (11h30m40.563317754s) [upload=11956]: Done starting upload Aug 7 21:10:18.819487043 (11h30m40.563543905s) [upload=11956]: Uploading segments... Aug 7 21:10:18.819584609 (11h30m40.563641481s) [upload=11956]: Got next segment. Inline: true Aug 7 21:10:18.819614709 (11h30m40.563671559s) [upload=11956]: Waiting for error group managing segments... Aug 7 21:10:18.819668987 (11h30m40.563725851s) [upload=11956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:10:18.819689667 (11h30m40.563746534s) [upload=11956]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:10:18.819694129 (11h30m40.563750997s) [upload=11956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:10:18.958211137 (11h30m40.702268007s) [upload=11956]: Done uploading segments... Aug 7 21:10:18.958235626 (11h30m40.702292473s) [upload=11956]: Upload finished. err: <nil> Aug 7 21:11:16.699978190 (11h31m38.444035039s) [upload=11969]: Starting upload Aug 7 21:11:16.700090349 (11h31m38.44414722s) [upload=11969]: Done starting upload Aug 7 21:11:16.700171865 (11h31m38.444228712s) [upload=11969]: Uploading segments... Aug 7 21:11:16.700229408 (11h31m38.444286283s) [upload=11969]: Got next segment. Inline: true Aug 7 21:11:16.700236785 (11h31m38.444293651s) [upload=11969]: Waiting for error group managing segments... Aug 7 21:11:16.700303583 (11h31m38.444360444s) [upload=11969]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:11:16.700311256 (11h31m38.444368121s) [upload=11969]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:16.700316036 (11h31m38.444372904s) [upload=11969]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:11:16.845168013 (11h31m38.589224864s) [upload=11969]: Done uploading segments... Aug 7 21:11:16.845187939 (11h31m38.5892448s) [upload=11969]: Upload finished. err: <nil> Aug 7 21:12:04.147595666 (11h32m25.891652543s) [upload=11983]: Starting upload Aug 7 21:12:04.147723257 (11h32m25.891780138s) [upload=11983]: Done starting upload Aug 7 21:12:04.147862477 (11h32m25.891919358s) [upload=11983]: Uploading segments... Aug 7 21:12:04.147924428 (11h32m25.891981303s) [upload=11983]: Got next segment. Inline: true Aug 7 21:12:04.147931028 (11h32m25.89198791s) [upload=11983]: Waiting for error group managing segments... Aug 7 21:12:04.147990094 (11h32m25.892046969s) [upload=11983]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:12:04.147996570 (11h32m25.892053447s) [upload=11983]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:04.148003467 (11h32m25.892060346s) [upload=11983]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:12:04.368229522 (11h32m26.112286373s) [upload=11983]: Done uploading segments... Aug 7 21:12:04.368249120 (11h32m26.11230597s) [upload=11983]: Upload finished. err: <nil> Aug 7 21:12:47.792015773 (11h33m9.536072646s) [upload=11992]: Starting upload Aug 7 21:12:47.792127865 (11h33m9.536184732s) [upload=11992]: Done starting upload Aug 7 21:12:47.792248444 (11h33m9.536305307s) [upload=11992]: Uploading segments... Aug 7 21:12:47.792351067 (11h33m9.536407934s) [upload=11992]: Got next segment. Inline: true Aug 7 21:12:47.792359617 (11h33m9.536416486s) [upload=11992]: Waiting for error group managing segments... Aug 7 21:12:47.792414618 (11h33m9.536471489s) [upload=11992]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:12:47.792420367 (11h33m9.536477238s) [upload=11992]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:47.792423893 (11h33m9.536480763s) [upload=11992]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:12:47.975624008 (11h33m9.71968088s) [upload=11992]: Done uploading segments... Aug 7 21:12:47.975648860 (11h33m9.71970573s) [upload=11992]: Upload finished. err: <nil> Aug 7 21:14:01.226859239 (11h34m22.970916111s) [upload=12013]: Starting upload Aug 7 21:14:01.227003646 (11h34m22.971060518s) [upload=12013]: Done starting upload Aug 7 21:14:01.227141194 (11h34m22.971198047s) [upload=12013]: Uploading segments... Aug 7 21:14:01.227219219 (11h34m22.971276084s) [upload=12013]: Got next segment. Inline: true Aug 7 21:14:01.227227170 (11h34m22.971284041s) [upload=12013]: Waiting for error group managing segments... Aug 7 21:14:01.227295298 (11h34m22.971352162s) [upload=12013]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:14:01.227301580 (11h34m22.971358443s) [upload=12013]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:01.227305300 (11h34m22.971362163s) [upload=12013]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:15:00.953618154 (11h35m22.697675004s) [upload=12013]: Done uploading segments... Aug 7 21:15:00.953640566 (11h35m22.697697426s) [upload=12013]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:01.394116473 (11h35m23.138173339s) [upload=12030]: Starting upload Aug 7 21:15:01.394242598 (11h35m23.138299459s) [upload=12030]: Done starting upload Aug 7 21:15:01.394373467 (11h35m23.138430318s) [upload=12030]: Uploading segments... Aug 7 21:15:01.394459055 (11h35m23.138515928s) [upload=12030]: Got next segment. Inline: true Aug 7 21:15:01.394467128 (11h35m23.138523996s) [upload=12030]: Waiting for error group managing segments... Aug 7 21:15:01.394548041 (11h35m23.138604915s) [upload=12030]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:15:01.394554816 (11h35m23.13861169s) [upload=12030]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:01.394559518 (11h35m23.138616392s) [upload=12030]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:15:03.584668408 (11h35m25.328725286s) [upload=12030]: Done uploading segments... Aug 7 21:15:03.584713689 (11h35m25.32877056s) [upload=12030]: Upload finished. err: <nil> Aug 7 21:15:54.636790163 (11h36m16.380847013s) [upload=12044]: Starting upload Aug 7 21:15:54.636877707 (11h36m16.380934638s) [upload=12044]: Done starting upload Aug 7 21:15:54.636936350 (11h36m16.3809932s) [upload=12044]: Uploading segments... Aug 7 21:15:54.636993824 (11h36m16.381050685s) [upload=12044]: Got next segment. Inline: true Aug 7 21:15:54.636999384 (11h36m16.381056244s) [upload=12044]: Waiting for error group managing segments... Aug 7 21:15:54.637044877 (11h36m16.381101727s) [upload=12044]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:15:54.637050662 (11h36m16.381107522s) [upload=12044]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:54.637053905 (11h36m16.381110766s) [upload=12044]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:54.359175610 (11h37m16.103232476s) [upload=12044]: Done uploading segments... Aug 7 21:16:54.359203156 (11h37m16.103260018s) [upload=12044]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:54.796260076 (11h37m16.54031694s) [upload=12060]: Starting upload Aug 7 21:16:54.796405798 (11h37m16.540462658s) [upload=12060]: Done starting upload Aug 7 21:16:54.796514557 (11h37m16.540571407s) [upload=12060]: Uploading segments... Aug 7 21:16:54.796581012 (11h37m16.540637875s) [upload=12060]: Got next segment. Inline: true Aug 7 21:16:54.796586298 (11h37m16.540643158s) [upload=12060]: Waiting for error group managing segments... Aug 7 21:16:54.796657533 (11h37m16.540714383s) [upload=12060]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:54.796663030 (11h37m16.540719894s) [upload=12060]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:54.796669424 (11h37m16.540726289s) [upload=12060]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:54.519327140 (11h38m16.263384004s) [upload=12060]: Done uploading segments... Aug 7 21:17:54.519368022 (11h38m16.263424882s) [upload=12060]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:54.944498267 (11h38m16.688555136s) [upload=12074]: Starting upload Aug 7 21:17:54.944617406 (11h38m16.688674267s) [upload=12074]: Done starting upload Aug 7 21:17:54.944695430 (11h38m16.688752285s) [upload=12074]: Uploading segments... Aug 7 21:17:54.944771865 (11h38m16.688828736s) [upload=12074]: Got next segment. Inline: true Aug 7 21:17:54.944781252 (11h38m16.688838123s) [upload=12074]: Waiting for error group managing segments... Aug 7 21:17:54.944841042 (11h38m16.688897916s) [upload=12074]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:54.944848212 (11h38m16.688905079s) [upload=12074]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:54.944851963 (11h38m16.688908827s) [upload=12074]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:54.673864467 (11h39m16.417921315s) [upload=12074]: Done uploading segments... Aug 7 21:18:54.673885859 (11h39m16.417942708s) [upload=12074]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m0.000726199s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration:
30.000846595s
FAIL (2025-08-07 21:18:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000879564s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
8 out of 60 is failed (threshold=3) [...................................................XX.XXXXXX] duration:
30.00126963s
FAIL (2025-08-07 21:18:41 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 3 (11h32m6.980294735s) [upload=11978]: Done starting upload Aug 7 21:11:45.236348119 (11h32m6.980404981s) [upload=11978]: Uploading segments... Aug 7 21:11:45.236413753 (11h32m6.980470623s) [upload=11978]: Got next segment. Inline: true Aug 7 21:11:45.236420317 (11h32m6.980477184s) [upload=11978]: Waiting for error group managing segments... Aug 7 21:11:45.236488708 (11h32m6.980545572s) [upload=11978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:11:45.236497667 (11h32m6.980554538s) [upload=11978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:45.236502593 (11h32m6.980559463s) [upload=11978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:11:45.401183978 (11h32m7.145240838s) [upload=11978]: Done uploading segments... Aug 7 21:11:45.401202859 (11h32m7.145259707s) [upload=11978]: Upload finished. err: <nil> Aug 7 21:12:53.894047478 (11h33m15.63810433s) [upload=11996]: Starting upload Aug 7 21:12:53.894132938 (11h33m15.638189798s) [upload=11996]: Done starting upload Aug 7 21:12:53.894212135 (11h33m15.638268984s) [upload=11996]: Uploading segments... Aug 7 21:12:53.894264850 (11h33m15.638321716s) [upload=11996]: Got next segment. Inline: true Aug 7 21:12:53.894270475 (11h33m15.638327327s) [upload=11996]: Waiting for error group managing segments... Aug 7 21:12:53.894325009 (11h33m15.638381859s) [upload=11996]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:12:53.894330697 (11h33m15.638387548s) [upload=11996]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:53.894334066 (11h33m15.638390917s) [upload=11996]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:13:23.625409415 (11h33m45.36946628s) [upload=11996]: Done uploading segments... Aug 7 21:13:23.625434285 (11h33m45.369491153s) [upload=11996]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:13:48.015197539 (11h34m9.7592544s) [upload=12011]: Starting upload Aug 7 21:13:48.015280384 (11h34m9.759337244s) [upload=12011]: Done starting upload Aug 7 21:13:48.015414624 (11h34m9.759471489s) [upload=12011]: Uploading segments... Aug 7 21:13:48.015556587 (11h34m9.75961346s) [upload=12011]: Got next segment. Inline: true Aug 7 21:13:48.015566646 (11h34m9.759623517s) [upload=12011]: Waiting for error group managing segments... Aug 7 21:13:48.015627984 (11h34m9.759684854s) [upload=12011]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:13:48.015635040 (11h34m9.75969191s) [upload=12011]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:48.015638442 (11h34m9.759695306s) [upload=12011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:14:17.742337869 (11h34m39.486394729s) [upload=12011]: Done uploading segments... Aug 7 21:14:17.742364963 (11h34m39.486421823s) [upload=12011]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:14:18.164815368 (11h34m39.908872221s) [upload=12020]: Starting upload Aug 7 21:14:18.164900260 (11h34m39.908957119s) [upload=12020]: Done starting upload Aug 7 21:14:18.164985120 (11h34m39.909041981s) [upload=12020]: Uploading segments... Aug 7 21:14:18.165076859 (11h34m39.909133729s) [upload=12020]: Got next segment. Inline: true Aug 7 21:14:18.165084880 (11h34m39.909141745s) [upload=12020]: Waiting for error group managing segments... Aug 7 21:14:18.165136947 (11h34m39.909193807s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:14:18.165142336 (11h34m39.909199196s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:18.165145871 (11h34m39.909202731s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:14:18.323816200 (11h34m40.067873054s) [upload=12020]: Done uploading segments... Aug 7 21:14:18.323839479 (11h34m40.067896328s) [upload=12020]: Upload finished. err: <nil> Aug 7 21:15:30.409753622 (11h35m52.153810483s) [upload=12036]: Starting upload Aug 7 21:15:30.409818667 (11h35m52.153875531s) [upload=12036]: Done starting upload Aug 7 21:15:30.409895806 (11h35m52.153952669s) [upload=12036]: Uploading segments... Aug 7 21:15:30.409994454 (11h35m52.154051324s) [upload=12036]: Got next segment. Inline: true Aug 7 21:15:30.410001467 (11h35m52.154058337s) [upload=12036]: Waiting for error group managing segments... Aug 7 21:15:30.410069055 (11h35m52.154125913s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:15:30.410077097 (11h35m52.154133959s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:30.410082368 (11h35m52.154139256s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:00.139341290 (11h36m21.883398189s) [upload=12036]: Done uploading segments... Aug 7 21:16:00.139580695 (11h36m21.88363756s) [upload=12036]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:00.566884400 (11h36m22.310941251s) [upload=12047]: Starting upload Aug 7 21:16:00.566993382 (11h36m22.311050244s) [upload=12047]: Done starting upload Aug 7 21:16:00.567095627 (11h36m22.311152478s) [upload=12047]: Uploading segments... Aug 7 21:16:00.567233167 (11h36m22.311290039s) [upload=12047]: Got next segment. Inline: true Aug 7 21:16:00.567240534 (11h36m22.311297398s) [upload=12047]: Waiting for error group managing segments... Aug 7 21:16:00.567303473 (11h36m22.311360333s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:00.567314947 (11h36m22.311371807s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:00.567322295 (11h36m22.311379156s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:30.298703441 (11h36m52.0427603s) [upload=12047]: Done uploading segments... Aug 7 21:16:30.298732421 (11h36m52.04278928s) [upload=12047]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:41.168277425 (11h37m2.912334286s) [upload=12055]: Starting upload Aug 7 21:16:41.168334314 (11h37m2.912391183s) [upload=12055]: Done starting upload Aug 7 21:16:41.168407611 (11h37m2.912464477s) [upload=12055]: Uploading segments... Aug 7 21:16:41.168464727 (11h37m2.912521594s) [upload=12055]: Got next segment. Inline: true Aug 7 21:16:41.168470534 (11h37m2.912527404s) [upload=12055]: Waiting for error group managing segments... Aug 7 21:16:41.168520756 (11h37m2.912577617s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:41.168526439 (11h37m2.912583303s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:41.168529826 (11h37m2.91258669s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:10.898488630 (11h37m32.642545483s) [upload=12055]: Done uploading segments... Aug 7 21:17:10.898512884 (11h37m32.642569743s) [upload=12055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:11.314708421 (11h37m33.058765271s) [upload=12064]: Starting upload Aug 7 21:17:11.314772571 (11h37m33.058829423s) [upload=12064]: Done starting upload Aug 7 21:17:11.314846098 (11h37m33.05890295s) [upload=12064]: Uploading segments... Aug 7 21:17:11.314935220 (11h37m33.058992089s) [upload=12064]: Got next segment. Inline: true Aug 7 21:17:11.314940559 (11h37m33.058997419s) [upload=12064]: Waiting for error group managing segments... Aug 7 21:17:11.314984274 (11h37m33.059041125s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:11.314989343 (11h37m33.059046203s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:11.314992536 (11h37m33.059049396s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:41.043046082 (11h38m2.787102951s) [upload=12064]: Done uploading segments... Aug 7 21:17:41.043076101 (11h38m2.787132966s) [upload=12064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:41.498964594 (11h38m3.243021445s) [upload=12072]: Starting upload Aug 7 21:17:41.499038639 (11h38m3.243095509s) [upload=12072]: Done starting upload Aug 7 21:17:41.499118466 (11h38m3.243175318s) [upload=12072]: Uploading segments... Aug 7 21:17:41.499176424 (11h38m3.243233293s) [upload=12072]: Got next segment. Inline: true Aug 7 21:17:41.499181917 (11h38m3.243238777s) [upload=12072]: Waiting for error group managing segments... Aug 7 21:17:41.499236606 (11h38m3.243293472s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:41.499242465 (11h38m3.24329933s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:41.499245854 (11h38m3.243302721s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:11.231025141 (11h38m32.97508201s) [upload=12072]: Done uploading segments... Aug 7 21:18:11.231066796 (11h38m32.975123655s) [upload=12072]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:11.675991066 (11h38m33.420047932s) [upload=12078]: Starting upload Aug 7 21:18:11.676081621 (11h38m33.42013848s) [upload=12078]: Done starting upload Aug 7 21:18:11.676172293 (11h38m33.420229154s) [upload=12078]: Uploading segments... Aug 7 21:18:11.676235479 (11h38m33.420292347s) [upload=12078]: Got next segment. Inline: true Aug 7 21:18:11.676241644 (11h38m33.420298508s) [upload=12078]: Waiting for error group managing segments... Aug 7 21:18:11.676304521 (11h38m33.420361386s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:18:11.676310489 (11h38m33.420367352s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:11.676314037 (11h38m33.420370906s) [upload=12078]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:41.388891420 (11h39m3.132948271s) [upload=12078]: Done uploading segments... Aug 7 21:18:41.388921897 (11h39m3.132978749s) [upload=12078]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.001286457s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
7 out of 60 is failed (threshold=3) [...................................................X..XXXXXX] duration:
30.029208741s
FAIL (2025-08-07 21:18:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.029240985s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
1m0.000616184s
FAIL (2025-08-07 21:18:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000634958s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......X.......X.............................................] duration:
944.920959ms
FAIL (2025-08-07 21:18:32 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
68895 (10h54m20.003725754s) [upload=11347]: Done uploading segments...
Aug 7 20:33:58.259694049 (10h54m20.00375091s) [upload=11347]: Upload finished. err: <nil>
Aug 7 20:34:42.291300370 (10h55m4.035357231s) [upload=11357]: Starting upload
Aug 7 20:34:42.291392440 (10h55m4.035449315s) [upload=11357]: Done starting upload
Aug 7 20:34:42.291505939 (10h55m4.035562788s) [upload=11357]: Uploading segments...
Aug 7 20:34:42.291587798 (10h55m4.035644667s) [upload=11357]: Got next segment. Inline: true
Aug 7 20:34:42.291610428 (10h55m4.035667289s) [upload=11357]: Waiting for error group managing segments...
Aug 7 20:34:42.291660078 (10h55m4.035716927s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:34:42.291665407 (10h55m4.035722257s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:34:42.291668818 (10h55m4.035725679s) [upload=11357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:34:42.445169337 (10h55m4.189226185s) [upload=11357]: Done uploading segments...
Aug 7 20:34:42.445187662 (10h55m4.189244508s) [upload=11357]: Upload finished. err: <nil>
Aug 7 20:35:56.993979713 (10h56m18.738036563s) [upload=11378]: Starting upload
Aug 7 20:35:56.994042734 (10h56m18.738099594s) [upload=11378]: Done starting upload
Aug 7 20:35:56.994133564 (10h56m18.738190418s) [upload=11378]: Uploading segments...
Aug 7 20:35:56.994188836 (10h56m18.738245705s) [upload=11378]: Got next segment. Inline: true
Aug 7 20:35:56.994194253 (10h56m18.738251113s) [upload=11378]: Waiting for error group managing segments...
Aug 7 20:35:56.994255059 (10h56m18.738311921s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:35:56.994261721 (10h56m18.738318586s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:35:56.994266228 (10h56m18.73832309s) [upload=11378]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:35:57.196915243 (10h56m18.940972094s) [upload=11378]: Done uploading segments...
Aug 7 20:35:57.196933580 (10h56m18.940990431s) [upload=11378]: Upload finished. err: <nil>
Aug 7 20:36:48.383926585 (10h57m10.127983431s) [upload=11392]: Starting upload
Aug 7 20:36:48.383977279 (10h57m10.128034139s) [upload=11392]: Done starting upload
Aug 7 20:36:48.384080928 (10h57m10.128137781s) [upload=11392]: Uploading segments...
Aug 7 20:36:48.384124913 (10h57m10.128181781s) [upload=11392]: Got next segment. Inline: true
Aug 7 20:36:48.384130188 (10h57m10.128187037s) [upload=11392]: Waiting for error group managing segments...
Aug 7 20:36:48.384183298 (10h57m10.128240148s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:36:48.384188890 (10h57m10.128245741s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:36:48.384192353 (10h57m10.128249203s) [upload=11392]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:36:48.597716681 (10h57m10.341773532s) [upload=11392]: Done uploading segments...
Aug 7 20:36:48.597735626 (10h57m10.341792476s) [upload=11392]: Upload finished. err: <nil>
Aug 7 20:37:59.886756865 (10h58m21.630813711s) [upload=11411]: Starting upload
Aug 7 20:37:59.886810662 (10h58m21.630867522s) [upload=11411]: Done starting upload
Aug 7 20:37:59.886870714 (10h58m21.630927561s) [upload=11411]: Uploading segments...
Aug 7 20:37:59.886906763 (10h58m21.630963634s) [upload=11411]: Got next segment. Inline: true
Aug 7 20:37:59.886911413 (10h58m21.630968259s) [upload=11411]: Waiting for error group managing segments...
Aug 7 20:37:59.886948286 (10h58m21.631005132s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:37:59.886952923 (10h58m21.631009769s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:37:59.886957250 (10h58m21.631014096s) [upload=11411]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:38:00.073123067 (10h58m21.817179918s) [upload=11411]: Done uploading segments...
Aug 7 20:38:00.073142248 (10h58m21.817199097s) [upload=11411]: Upload finished. err: <nil>
Aug 7 20:39:06.192304563 (10h59m27.936361423s) [upload=11429]: Starting upload
Aug 7 20:39:06.192379945 (10h59m27.936436804s) [upload=11429]: Done starting upload
Aug 7 20:39:06.192450925 (10h59m27.936507772s) [upload=11429]: Uploading segments...
Aug 7 20:39:06.192493866 (10h59m27.936550726s) [upload=11429]: Got next segment. Inline: true
Aug 7 20:39:06.192499400 (10h59m27.93655626s) [upload=11429]: Waiting for error group managing segments...
Aug 7 20:39:06.192549058 (10h59m27.936605918s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:39:06.192553658 (10h59m27.936610518s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:39:06.192556283 (10h59m27.936613143s) [upload=11429]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:39:06.410732058 (10h59m28.154788918s) [upload=11429]: Done uploading segments...
Aug 7 20:39:06.410751072 (10h59m28.154807921s) [upload=11429]: Upload finished. err: <nil>
Aug 7 20:39:38.189437550 (10h59m59.9334944s) [upload=11438]: Starting upload
Aug 7 20:39:38.189494699 (10h59m59.933551565s) [upload=11438]: Done starting upload
Aug 7 20:39:38.189570885 (10h59m59.933627735s) [upload=11438]: Uploading segments...
Aug 7 20:39:38.189613828 (10h59m59.933670698s) [upload=11438]: Got next segment. Inline: true
Aug 7 20:39:38.189618898 (10h59m59.933675758s) [upload=11438]: Waiting for error group managing segments...
Aug 7 20:39:38.189671547 (10h59m59.933728396s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:39:38.189676923 (10h59m59.933733774s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:39:38.189680363 (10h59m59.933737214s) [upload=11438]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:39:38.339370287 (11h0m0.083427138s) [upload=11438]: Done uploading segments...
Aug 7 20:39:38.339394965 (11h0m0.083451822s) [upload=11438]: Upload finished. err: <nil>
Aug 7 20:40:13.049368895 (11h0m34.793425745s) [upload=11448]: Starting upload
Aug 7 20:40:13.049412890 (11h0m34.793469743s) [upload=11448]: Done starting upload
Aug 7 20:40:13.049490173 (11h0m34.793547027s) [upload=11448]: Uploading segments...
Aug 7 20:40:13.049531553 (11h0m34.793588423s) [upload=11448]: Got next segment. Inline: true
Aug 7 20:40:13.049536305 (11h0m34.79359316s) [upload=11448]: Waiting for error group managing segments...
Aug 7 20:40:13.049606089 (11h0m34.793662937s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:40:13.049611863 (11h0m34.793668715s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:40:13.049615487 (11h0m34.793672338s) [upload=11448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:40:13.251485869 (11h0m34.995542721s) [upload=11448]: Done uploading segments...
Aug 7 20:40:13.251504252 (11h0m34.995561102s) [upload=11448]: Upload finished. err: <nil>
Aug 7 20:41:01.300323709 (11h1m23.044380561s) [upload=11462]: Starting upload
Aug 7 20:41:01.300386685 (11h1m23.044443554s) [upload=11462]: Done starting upload
Aug 7 20:41:01.300471424 (11h1m23.044528277s) [upload=11462]: Uploading segments...
Aug 7 20:41:01.300528867 (11h1m23.044585736s) [upload=11462]: Got next segment. Inline: true
Aug 7 20:41:01.300534426 (11h1m23.044591286s) [upload=11462]: Waiting for error group managing segments...
Aug 7 20:41:01.300592418 (11h1m23.044649268s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:01.300598189 (11h1m23.04465505s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:01.300601616 (11h1m23.044658476s) [upload=11462]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:01.448083138 (11h1m23.192139989s) [upload=11462]: Done uploading segments...
Aug 7 20:41:01.448102791 (11h1m23.192159644s) [upload=11462]: Upload finished. err: <nil>
Aug 7 20:41:25.522931110 (11h1m47.266987978s) [upload=11468]: Starting upload
Aug 7 20:41:25.523039051 (11h1m47.267095924s) [upload=11468]: Done starting upload
Aug 7 20:41:25.523181759 (11h1m47.267238621s) [upload=11468]: Uploading segments...
Aug 7 20:41:25.523403073 (11h1m47.267459944s) [upload=11468]: Got next segment. Inline: true
Aug 7 20:41:25.523413059 (11h1m47.26746993s) [upload=11468]: Waiting for error group managing segments...
Aug 7 20:41:25.523492519 (11h1m47.267549389s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:25.523498700 (11h1m47.267555566s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:25.523502761 (11h1m47.267559629s) [upload=11468]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:25.840530122 (11h1m47.584586983s) [upload=11468]: Done uploading segments...
Aug 7 20:41:25.840544406 (11h1m47.584601253s) [upload=11468]: Upload finished. err: <nil>
Aug 7 20:41:49.037046771 (11h2m10.781103621s) [upload=11474]: Starting upload
Aug 7 20:41:49.037108376 (11h2m10.781165228s) [upload=11474]: Done starting upload
Aug 7 20:41:49.037181045 (11h2m10.781237895s) [upload=11474]: Uploading segments...
Aug 7 20:41:49.037238734 (11h2m10.781295606s) [upload=11474]: Got next segment. Inline: true
Aug 7 20:41:49.037244163 (11h2m10.781301028s) [upload=11474]: Waiting for error group managing segments...
Aug 7 20:41:49.037309765 (11h2m10.781366628s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 20:41:49.037314702 (11h2m10.781371562s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 7 20:41:49.037317447 (11h2m10.781374307s) [upload=11474]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 20:41:49.205260584 (11h2m10.949317434s) [upload=11474]: Done uploading segments...
Aug 7 20:41:49.205278299 (11h2m10.94933515s) [upload=11474]: Upload finished. err: <nil>
Duration: 944.955696ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [................................X......................X....] duration:
2.841210669s
FAIL (2025-08-07 21:18:19 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
oad finished. err: <nil>
Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload
Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload
Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments...
Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false
Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment...
Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment.
Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece...
Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment
Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments...
Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil>
Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces.
Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF
Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments...
Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil>
Duration: 2.841238997s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
7 out of 60 is failed (threshold=3) [....................................................XX.XXXXX] duration:
30.001343423s
FAIL (2025-08-07 21:18:11 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload Aug 7 21:11:26.616528589 (11h31m48.36058546s) [upload=11970]: Done starting upload Aug 7 21:11:26.616620452 (11h31m48.360677314s) [upload=11970]: Uploading segments... Aug 7 21:11:26.616680665 (11h31m48.360737537s) [upload=11970]: Got next segment. Inline: true Aug 7 21:11:26.616687208 (11h31m48.360744077s) [upload=11970]: Waiting for error group managing segments... Aug 7 21:11:26.616741764 (11h31m48.360798633s) [upload=11970]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:11:26.616747736 (11h31m48.360804606s) [upload=11970]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:26.616751310 (11h31m48.360808175s) [upload=11970]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:11:26.789391288 (11h31m48.533448153s) [upload=11970]: Done uploading segments... Aug 7 21:11:26.789430240 (11h31m48.533487103s) [upload=11970]: Upload finished. err: <nil> Aug 7 21:11:45.236119686 (11h32m6.980176554s) [upload=11978]: Starting upload Aug 7 21:11:45.236234403 (11h32m6.980294735s) [upload=11978]: Done starting upload Aug 7 21:11:45.236348119 (11h32m6.980404981s) [upload=11978]: Uploading segments... Aug 7 21:11:45.236413753 (11h32m6.980470623s) [upload=11978]: Got next segment. Inline: true Aug 7 21:11:45.236420317 (11h32m6.980477184s) [upload=11978]: Waiting for error group managing segments... Aug 7 21:11:45.236488708 (11h32m6.980545572s) [upload=11978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:11:45.236497667 (11h32m6.980554538s) [upload=11978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:45.236502593 (11h32m6.980559463s) [upload=11978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:11:45.401183978 (11h32m7.145240838s) [upload=11978]: Done uploading segments... Aug 7 21:11:45.401202859 (11h32m7.145259707s) [upload=11978]: Upload finished. err: <nil> Aug 7 21:12:53.894047478 (11h33m15.63810433s) [upload=11996]: Starting upload Aug 7 21:12:53.894132938 (11h33m15.638189798s) [upload=11996]: Done starting upload Aug 7 21:12:53.894212135 (11h33m15.638268984s) [upload=11996]: Uploading segments... Aug 7 21:12:53.894264850 (11h33m15.638321716s) [upload=11996]: Got next segment. Inline: true Aug 7 21:12:53.894270475 (11h33m15.638327327s) [upload=11996]: Waiting for error group managing segments... Aug 7 21:12:53.894325009 (11h33m15.638381859s) [upload=11996]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:12:53.894330697 (11h33m15.638387548s) [upload=11996]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:53.894334066 (11h33m15.638390917s) [upload=11996]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:13:23.625409415 (11h33m45.36946628s) [upload=11996]: Done uploading segments... Aug 7 21:13:23.625434285 (11h33m45.369491153s) [upload=11996]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:13:48.015197539 (11h34m9.7592544s) [upload=12011]: Starting upload Aug 7 21:13:48.015280384 (11h34m9.759337244s) [upload=12011]: Done starting upload Aug 7 21:13:48.015414624 (11h34m9.759471489s) [upload=12011]: Uploading segments... Aug 7 21:13:48.015556587 (11h34m9.75961346s) [upload=12011]: Got next segment. Inline: true Aug 7 21:13:48.015566646 (11h34m9.759623517s) [upload=12011]: Waiting for error group managing segments... Aug 7 21:13:48.015627984 (11h34m9.759684854s) [upload=12011]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:13:48.015635040 (11h34m9.75969191s) [upload=12011]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:48.015638442 (11h34m9.759695306s) [upload=12011]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:14:17.742337869 (11h34m39.486394729s) [upload=12011]: Done uploading segments... Aug 7 21:14:17.742364963 (11h34m39.486421823s) [upload=12011]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:14:18.164815368 (11h34m39.908872221s) [upload=12020]: Starting upload Aug 7 21:14:18.164900260 (11h34m39.908957119s) [upload=12020]: Done starting upload Aug 7 21:14:18.164985120 (11h34m39.909041981s) [upload=12020]: Uploading segments... Aug 7 21:14:18.165076859 (11h34m39.909133729s) [upload=12020]: Got next segment. Inline: true Aug 7 21:14:18.165084880 (11h34m39.909141745s) [upload=12020]: Waiting for error group managing segments... Aug 7 21:14:18.165136947 (11h34m39.909193807s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:14:18.165142336 (11h34m39.909199196s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:18.165145871 (11h34m39.909202731s) [upload=12020]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:14:18.323816200 (11h34m40.067873054s) [upload=12020]: Done uploading segments... Aug 7 21:14:18.323839479 (11h34m40.067896328s) [upload=12020]: Upload finished. err: <nil> Aug 7 21:15:30.409753622 (11h35m52.153810483s) [upload=12036]: Starting upload Aug 7 21:15:30.409818667 (11h35m52.153875531s) [upload=12036]: Done starting upload Aug 7 21:15:30.409895806 (11h35m52.153952669s) [upload=12036]: Uploading segments... Aug 7 21:15:30.409994454 (11h35m52.154051324s) [upload=12036]: Got next segment. Inline: true Aug 7 21:15:30.410001467 (11h35m52.154058337s) [upload=12036]: Waiting for error group managing segments... Aug 7 21:15:30.410069055 (11h35m52.154125913s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:15:30.410077097 (11h35m52.154133959s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:30.410082368 (11h35m52.154139256s) [upload=12036]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:00.139341290 (11h36m21.883398189s) [upload=12036]: Done uploading segments... Aug 7 21:16:00.139580695 (11h36m21.88363756s) [upload=12036]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:00.566884400 (11h36m22.310941251s) [upload=12047]: Starting upload Aug 7 21:16:00.566993382 (11h36m22.311050244s) [upload=12047]: Done starting upload Aug 7 21:16:00.567095627 (11h36m22.311152478s) [upload=12047]: Uploading segments... Aug 7 21:16:00.567233167 (11h36m22.311290039s) [upload=12047]: Got next segment. Inline: true Aug 7 21:16:00.567240534 (11h36m22.311297398s) [upload=12047]: Waiting for error group managing segments... Aug 7 21:16:00.567303473 (11h36m22.311360333s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:00.567314947 (11h36m22.311371807s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:00.567322295 (11h36m22.311379156s) [upload=12047]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:16:30.298703441 (11h36m52.0427603s) [upload=12047]: Done uploading segments... Aug 7 21:16:30.298732421 (11h36m52.04278928s) [upload=12047]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:41.168277425 (11h37m2.912334286s) [upload=12055]: Starting upload Aug 7 21:16:41.168334314 (11h37m2.912391183s) [upload=12055]: Done starting upload Aug 7 21:16:41.168407611 (11h37m2.912464477s) [upload=12055]: Uploading segments... Aug 7 21:16:41.168464727 (11h37m2.912521594s) [upload=12055]: Got next segment. Inline: true Aug 7 21:16:41.168470534 (11h37m2.912527404s) [upload=12055]: Waiting for error group managing segments... Aug 7 21:16:41.168520756 (11h37m2.912577617s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:16:41.168526439 (11h37m2.912583303s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:41.168529826 (11h37m2.91258669s) [upload=12055]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:10.898488630 (11h37m32.642545483s) [upload=12055]: Done uploading segments... Aug 7 21:17:10.898512884 (11h37m32.642569743s) [upload=12055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:11.314708421 (11h37m33.058765271s) [upload=12064]: Starting upload Aug 7 21:17:11.314772571 (11h37m33.058829423s) [upload=12064]: Done starting upload Aug 7 21:17:11.314846098 (11h37m33.05890295s) [upload=12064]: Uploading segments... Aug 7 21:17:11.314935220 (11h37m33.058992089s) [upload=12064]: Got next segment. Inline: true Aug 7 21:17:11.314940559 (11h37m33.058997419s) [upload=12064]: Waiting for error group managing segments... Aug 7 21:17:11.314984274 (11h37m33.059041125s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:11.314989343 (11h37m33.059046203s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:11.314992536 (11h37m33.059049396s) [upload=12064]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:17:41.043046082 (11h38m2.787102951s) [upload=12064]: Done uploading segments... Aug 7 21:17:41.043076101 (11h38m2.787132966s) [upload=12064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:41.498964594 (11h38m3.243021445s) [upload=12072]: Starting upload Aug 7 21:17:41.499038639 (11h38m3.243095509s) [upload=12072]: Done starting upload Aug 7 21:17:41.499118466 (11h38m3.243175318s) [upload=12072]: Uploading segments... Aug 7 21:17:41.499176424 (11h38m3.243233293s) [upload=12072]: Got next segment. Inline: true Aug 7 21:17:41.499181917 (11h38m3.243238777s) [upload=12072]: Waiting for error group managing segments... Aug 7 21:17:41.499236606 (11h38m3.243293472s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 21:17:41.499242465 (11h38m3.24329933s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:41.499245854 (11h38m3.243302721s) [upload=12072]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 21:18:11.231025141 (11h38m32.97508201s) [upload=12072]: Done uploading segments... Aug 7 21:18:11.231066796 (11h38m32.975123655s) [upload=12072]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.001362955s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration:
30.000870417s
FAIL (2025-08-07 21:18:08 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: context deadline exceeded
======LOGS======
11h33m54.620403019s) [noise=true , address=104.36.82.14:4223 , piece_id=PGFS7PTR7FMWOGLQ5LLSCYL25Y7TKZTKHEL323CD6VTNEKXRBHPQ, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.891850619 (11h33m54.63590748s) [noise=true , address=45.91.116.13:4154 , piece_id=MB4VFURZEYXUI7RPS4ULUETD6XKG3USRD74F2WFOM7WZ7QUNF6LA, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.894779023 (11h33m54.638835953s) [noise=true , address=45.91.116.9:4262 , piece_id=GYWGFWV4C3PR4RUYF32MAQRNWKWVPZLRO2SO4UJCJK65R23JUZGQ, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.917747403 (11h33m54.661804264s) [noise=true , address=45.91.116.10:4083 , piece_id=X2HPSGJZ2A5QROSCDSCA7RTFV7JQPTGCDRWYQZWAKXUOXKGXVQNA, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.924108489 (11h33m54.668165335s) [noise=true , address=45.91.116.8:4045 , piece_id=2NUYYQ3WGF2UO2ROSY3R6FN43YEYFEPHGAK6HHV4MXNZIAUCB6JA, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.936065869 (11h33m54.680122736s) [noise=true , address=77.90.31.12:4589 , piece_id=DQLSJRXEL4SG63R6DH6YPSKBRFCCDZ2M6VS5NX2X72LKQ4XMHH6A, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.943684467 (11h33m54.687741333s) [noise=true , address=45.91.116.13:4153 , piece_id=VCQLYREVBH3VTQVVQ4N6RJJKIA5RTPLROMUZKFH62A3P7B4GLFTQ, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.969180656 (11h33m54.713237508s) [noise=true , address=77.90.31.19:4843 , piece_id=3FPX5CXVNG7IFODGUVUYL3V6RECWAGN7SYDS2MXEUE6FECWWCHZA, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.986237351 (11h33m54.730294197s) [noise=true , address=45.91.116.11:4117 , piece_id=TH2ZUH4NO4YPRUNW4ABP2BLBXEVQB2KVEJEK3COYPQGMG3AISQBQ, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.986598715 (11h33m54.730655567s) [noise=true , address=77.90.31.13:4628 , piece_id=5L2DJMNTXMXQC4NNUW742I4RYY5U4JZCW66MUW4C7R6A3IJ46WHA, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:32.993716558 (11h33m54.73777342s) [noise=true , address=77.90.31.15:4699 , piece_id=4QFC6XUXFSCGNCVUQ6APWFP4VPOT2XEPBF6MLLNJPWJIAHWTNAQQ, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:33.004764885 (11h33m54.748821753s) [noise=true , address=45.91.116.12:4225 , piece_id=WBWYO5U3GRZX4MAS5TVYV4X75LM2IM4MYAALYWITSTJY3G6C6V3A, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:33.007513393 (11h33m54.75157026s) [noise=true , address=104.36.82.11:4114 , piece_id=CZPJM2V75OCGBKDT3VONHMK4UK7UV7W5CM7ALZV63Z5WA44R5KHA, seg_pos={0 0}, upload=12005]: Done uploading piece. err: <nil>
Aug 7 21:13:33.007550343 (11h33m54.751607205s) [seg_pos={0 0}, upload=12005]: Segment reached optimal threshold of 13 pieces.
Aug 7 21:13:33.007818213 (11h33m54.75187508s) [noise=true , address=104.36.82.9:4006 , piece_id=DUDTTETXBUSMAJP7HWVVVU6JNB3NODJEYNKCWJWVIZYACCPF7KIQ, seg_pos={0 0}, upload=12005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: expected piece hash; context canceled; EOF
Aug 7 21:13:33.007957507 (11h33m54.752014368s) [noise=true , address=45.91.116.9:4263 , piece_id=U4WUKRSZCL757XKP25KUR6CQ7GNX2NQ544SBF7DEP4DLKDLQCT5A, seg_pos={0 0}, upload=12005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF
Aug 7 21:13:33.008072807 (11h33m54.752129674s) [noise=true , address=45.91.116.10:4082 , piece_id=OOR4QA3CXAA7G4AJRO2MJVKKAPWNMWXEG4MGPHFSSEJJG2PAP3MA, seg_pos={0 0}, upload=12005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF
Aug 7 21:13:33.008088785 (11h33m54.752145655s) [seg_pos={0 0}, upload=12005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 7 21:13:33.008239833 (11h33m54.752296703s) [upload=12005]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 21:13:33.008246480 (11h33m54.752303365s) [upload=12005]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 21:13:56.682120899 (11h34m18.426177749s) [upload=12005]: Done uploading segments...
Aug 7 21:13:56.682153262 (11h34m18.426210125s) [upload=12005]: Upload finished. err: metaclient: context deadline exceeded
Aug 7 21:14:08.981143212 (11h34m30.725200099s) [upload=12017]: Starting upload
Aug 7 21:14:08.981247915 (11h34m30.725304782s) [upload=12017]: Done starting upload
Aug 7 21:14:08.981457434 (11h34m30.725514297s) [upload=12017]: Uploading segments...
Aug 7 21:14:08.981983258 (11h34m30.726040126s) [upload=12017]: Got next segment. Inline: false
Aug 7 21:14:08.982063476 (11h34m30.726120349s) [upload=12017]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:14:08.982070288 (11h34m30.726127163s) [upload=12017]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:14:38.713635311 (11h35m0.457692174s) [upload=12017]: Next segment err: context deadline exceeded
Aug 7 21:14:38.713818148 (11h35m0.457874995s) [upload=12017]: Done uploading segments...
Aug 7 21:14:38.713847460 (11h35m0.457904307s) [upload=12017]: Upload finished. err: context deadline exceeded
Aug 7 21:14:39.126797864 (11h35m0.870854713s) [upload=12024]: Starting upload
Aug 7 21:14:39.126867479 (11h35m0.87092433s) [upload=12024]: Done starting upload
Aug 7 21:14:39.127134096 (11h35m0.871190958s) [upload=12024]: Uploading segments...
Aug 7 21:14:39.127206037 (11h35m0.871262906s) [upload=12024]: Got next segment. Inline: false
Aug 7 21:14:39.127247021 (11h35m0.871303875s) [upload=12024]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:14:39.127254726 (11h35m0.871311587s) [upload=12024]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:15:08.857744672 (11h35m30.601801542s) [upload=12024]: Next segment err: context deadline exceeded
Aug 7 21:15:08.857937199 (11h35m30.60199407s) [upload=12024]: Done uploading segments...
Aug 7 21:15:08.857962321 (11h35m30.602019169s) [upload=12024]: Upload finished. err: context deadline exceeded
Aug 7 21:15:20.485893561 (11h35m42.229950429s) [upload=12034]: Starting upload
Aug 7 21:15:20.486023409 (11h35m42.230080283s) [upload=12034]: Done starting upload
Aug 7 21:15:20.486134808 (11h35m42.230191661s) [upload=12034]: Uploading segments...
Aug 7 21:15:20.486431156 (11h35m42.230488009s) [upload=12034]: Got next segment. Inline: false
Aug 7 21:15:20.486483996 (11h35m42.230540863s) [upload=12034]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:15:20.486491542 (11h35m42.230548409s) [upload=12034]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:15:50.215320677 (11h36m11.959377531s) [upload=12034]: Next segment err: context deadline exceeded
Aug 7 21:15:50.215551089 (11h36m11.95960795s) [upload=12034]: Done uploading segments...
Aug 7 21:15:50.215590354 (11h36m11.959647201s) [upload=12034]: Upload finished. err: context deadline exceeded
Aug 7 21:15:59.944633175 (11h36m21.688690025s) [upload=12045]: Starting upload
Aug 7 21:15:59.944712435 (11h36m21.6887693s) [upload=12045]: Done starting upload
Aug 7 21:15:59.945075324 (11h36m21.689132187s) [upload=12045]: Uploading segments...
Aug 7 21:15:59.945181160 (11h36m21.689238043s) [upload=12045]: Got next segment. Inline: false
Aug 7 21:15:59.945260803 (11h36m21.689317664s) [upload=12045]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:15:59.945267828 (11h36m21.68932469s) [upload=12045]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:16:29.659806113 (11h36m51.403862973s) [upload=12045]: Next segment err: context deadline exceeded
Aug 7 21:16:29.659892256 (11h36m51.403949103s) [upload=12045]: Done uploading segments...
Aug 7 21:16:29.659910327 (11h36m51.403967174s) [upload=12045]: Upload finished. err: context deadline exceeded
Aug 7 21:16:40.061254158 (11h37m1.80531101s) [upload=12054]: Starting upload
Aug 7 21:16:40.061330739 (11h37m1.805387599s) [upload=12054]: Done starting upload
Aug 7 21:16:40.061534219 (11h37m1.805591069s) [upload=12054]: Uploading segments...
Aug 7 21:16:40.061675268 (11h37m1.805732119s) [upload=12054]: Got next segment. Inline: false
Aug 7 21:16:40.061734809 (11h37m1.805791655s) [upload=12054]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:16:40.061740406 (11h37m1.805797266s) [upload=12054]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:17:09.783636482 (11h37m31.52769334s) [upload=12054]: Next segment err: context deadline exceeded
Aug 7 21:17:09.783809752 (11h37m31.527866609s) [upload=12054]: Done uploading segments...
Aug 7 21:17:09.783883524 (11h37m31.527940372s) [upload=12054]: Upload finished. err: context deadline exceeded
Aug 7 21:17:38.423613839 (11h38m0.16767071s) [upload=12070]: Starting upload
Aug 7 21:17:38.423678998 (11h38m0.167735863s) [upload=12070]: Done starting upload
Aug 7 21:17:38.423788045 (11h38m0.167844912s) [upload=12070]: Uploading segments...
Aug 7 21:17:38.423932717 (11h38m0.167989565s) [upload=12070]: Got next segment. Inline: false
Aug 7 21:17:38.424005392 (11h38m0.168062243s) [upload=12070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 21:17:38.424011139 (11h38m0.168068002s) [upload=12070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 21:18:08.145022440 (11h38m29.889079332s) [upload=12070]: Next segment err: context deadline exceeded
Aug 7 21:18:08.145219437 (11h38m29.889276305s) [upload=12070]: Done uploading segments...
Aug 7 21:18:08.145260561 (11h38m29.889317417s) [upload=12070]: Upload finished. err: context deadline exceeded
Duration: 30.000897177s
core
uplink
us-select-1
|