Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X......................X......................] duration: 3.22994456s FAIL (2025-08-07 21:31:12 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== oad finished. err: <nil> Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments... Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment... Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment. Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments... Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces. Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments... Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil> Duration: 3.22998026s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 17 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXXXXXXX] duration: 30.000697089s FAIL (2025-08-07 21:31:11 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000710595s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 24 out of 60 is failed (threshold=3) [...................................XX.XXXXXXXXXXXXXXXXXXXXXX] duration: 30.001154107s FAIL (2025-08-07 21:31:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 3m28.645134714s) [upload=12145]: Uploading segments... Aug 7 21:23:06.901186098 (11h43m28.645242968s) [upload=12145]: Got next segment. Inline: true Aug 7 21:23:06.901209675 (11h43m28.645266536s) [upload=12145]: Waiting for error group managing segments... Aug 7 21:23:06.901265651 (11h43m28.645322518s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:06.901287666 (11h43m28.645344528s) [upload=12145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:06.901291297 (11h43m28.645348161s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.622244822 (11h43m58.366301669s) [upload=12145]: Done uploading segments... Aug 7 21:23:36.622267267 (11h43m58.366324114s) [upload=12145]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:20.942800102 (11h44m42.686856953s) [upload=12164]: Starting upload Aug 7 21:24:20.942876168 (11h44m42.686933032s) [upload=12164]: Done starting upload Aug 7 21:24:20.942953347 (11h44m42.687010195s) [upload=12164]: Uploading segments... Aug 7 21:24:20.943004925 (11h44m42.687061797s) [upload=12164]: Got next segment. Inline: true Aug 7 21:24:20.943010718 (11h44m42.687067582s) [upload=12164]: Waiting for error group managing segments... Aug 7 21:24:20.943057728 (11h44m42.687114586s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:20.943063424 (11h44m42.687120287s) [upload=12164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:20.943066967 (11h44m42.687123828s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:50.669999700 (11h45m12.414056561s) [upload=12164]: Done uploading segments... Aug 7 21:24:50.670022022 (11h45m12.414078882s) [upload=12164]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:15.759998843 (11h45m37.504055689s) [upload=12180]: Starting upload Aug 7 21:25:15.760060918 (11h45m37.504117764s) [upload=12180]: Done starting upload Aug 7 21:25:15.760121258 (11h45m37.504178104s) [upload=12180]: Uploading segments... Aug 7 21:25:15.760159734 (11h45m37.504216604s) [upload=12180]: Got next segment. Inline: true Aug 7 21:25:15.760164169 (11h45m37.504221029s) [upload=12180]: Waiting for error group managing segments... Aug 7 21:25:15.760202724 (11h45m37.504259584s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:15.760207148 (11h45m37.504264007s) [upload=12180]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:15.760212108 (11h45m37.504268971s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:45.477193191 (11h46m7.22125006s) [upload=12180]: Done uploading segments... Aug 7 21:25:45.477222897 (11h46m7.22127976s) [upload=12180]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:45.889252639 (11h46m7.633309501s) [upload=12186]: Starting upload Aug 7 21:25:45.889337905 (11h46m7.633394765s) [upload=12186]: Done starting upload Aug 7 21:25:45.889442879 (11h46m7.633499729s) [upload=12186]: Uploading segments... Aug 7 21:25:45.889501484 (11h46m7.633558347s) [upload=12186]: Got next segment. Inline: true Aug 7 21:25:45.889507484 (11h46m7.633564344s) [upload=12186]: Waiting for error group managing segments... Aug 7 21:25:45.889551118 (11h46m7.633607968s) [upload=12186]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:45.889556440 (11h46m7.633613289s) [upload=12186]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:45.889559845 (11h46m7.633616705s) [upload=12186]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:15.619717489 (11h46m37.363774339s) [upload=12186]: Done uploading segments... Aug 7 21:26:15.619739739 (11h46m37.363796589s) [upload=12186]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:40.252456263 (11h47m1.996513132s) [upload=12201]: Starting upload Aug 7 21:26:40.252598168 (11h47m1.996655038s) [upload=12201]: Done starting upload Aug 7 21:26:40.252702071 (11h47m1.996758923s) [upload=12201]: Uploading segments... Aug 7 21:26:40.252782136 (11h47m1.996839003s) [upload=12201]: Got next segment. Inline: true Aug 7 21:26:40.252790092 (11h47m1.996846954s) [upload=12201]: Waiting for error group managing segments... Aug 7 21:26:40.252842463 (11h47m1.996899315s) [upload=12201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:40.252849740 (11h47m1.996906592s) [upload=12201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:40.252854153 (11h47m1.996911015s) [upload=12201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:09.981768433 (11h47m31.725825298s) [upload=12201]: Done uploading segments... Aug 7 21:27:09.981793631 (11h47m31.725850483s) [upload=12201]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:26.837217755 (11h47m48.581274619s) [upload=12211]: Starting upload Aug 7 21:27:26.837300223 (11h47m48.581357084s) [upload=12211]: Done starting upload Aug 7 21:27:26.837385984 (11h47m48.581442831s) [upload=12211]: Uploading segments... Aug 7 21:27:26.837444443 (11h47m48.581501313s) [upload=12211]: Got next segment. Inline: true Aug 7 21:27:26.837449770 (11h47m48.581506619s) [upload=12211]: Waiting for error group managing segments... Aug 7 21:27:26.837517257 (11h47m48.581574107s) [upload=12211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:26.837523154 (11h47m48.581580003s) [upload=12211]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:26.837527097 (11h47m48.58158395s) [upload=12211]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:56.564982463 (11h48m18.309039313s) [upload=12211]: Done uploading segments... Aug 7 21:27:56.565004020 (11h48m18.309060871s) [upload=12211]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:36.367399060 (11h48m58.11145591s) [upload=12226]: Starting upload Aug 7 21:28:36.367588554 (11h48m58.111645417s) [upload=12226]: Done starting upload Aug 7 21:28:36.367664912 (11h48m58.111721765s) [upload=12226]: Uploading segments... Aug 7 21:28:36.367723895 (11h48m58.111780757s) [upload=12226]: Got next segment. Inline: true Aug 7 21:28:36.367729331 (11h48m58.111786192s) [upload=12226]: Waiting for error group managing segments... Aug 7 21:28:36.367785894 (11h48m58.111842754s) [upload=12226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:36.367791010 (11h48m58.11184787s) [upload=12226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:36.367794492 (11h48m58.111851352s) [upload=12226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:06.092890267 (11h49m27.836947129s) [upload=12226]: Done uploading segments... Aug 7 21:29:06.092932205 (11h49m27.836989051s) [upload=12226]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:29:06.522310183 (11h49m28.266367033s) [upload=12233]: Starting upload Aug 7 21:29:06.522388772 (11h49m28.266445633s) [upload=12233]: Done starting upload Aug 7 21:29:06.522479144 (11h49m28.266535995s) [upload=12233]: Uploading segments... Aug 7 21:29:06.522560400 (11h49m28.266617269s) [upload=12233]: Got next segment. Inline: true Aug 7 21:29:06.522566995 (11h49m28.266623844s) [upload=12233]: Waiting for error group managing segments... Aug 7 21:29:06.522615251 (11h49m28.266672101s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:06.522621757 (11h49m28.266678608s) [upload=12233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:06.522630028 (11h49m28.266686879s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:36.250046349 (11h49m57.994103211s) [upload=12233]: Done uploading segments... Aug 7 21:29:36.250088941 (11h49m57.994145808s) [upload=12233]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:30:03.947527959 (11h50m25.691584822s) [upload=12249]: Starting upload Aug 7 21:30:03.947767439 (11h50m25.691824311s) [upload=12249]: Done starting upload Aug 7 21:30:03.959515214 (11h50m25.703572066s) [upload=12249]: Uploading segments... Aug 7 21:30:03.959659046 (11h50m25.703715962s) [upload=12249]: Got next segment. Inline: true Aug 7 21:30:03.959668426 (11h50m25.703725302s) [upload=12249]: Waiting for error group managing segments... Aug 7 21:30:03.959783210 (11h50m25.703840082s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:30:03.959790677 (11h50m25.703847531s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:30:03.959795320 (11h50m25.703852169s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:30:33.655073335 (11h50m55.399130195s) [upload=12249]: Done uploading segments... Aug 7 21:30:33.655096560 (11h50m55.399153412s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:30:38.130475962 (11h50m59.874532829s) [upload=12256]: Starting upload Aug 7 21:30:38.130573915 (11h50m59.874630787s) [upload=12256]: Done starting upload Aug 7 21:30:38.130665890 (11h50m59.874722739s) [upload=12256]: Uploading segments... Aug 7 21:30:38.131017420 (11h50m59.875074302s) [upload=12256]: Got next segment. Inline: true Aug 7 21:30:38.131029217 (11h50m59.87508609s) [upload=12256]: Waiting for error group managing segments... Aug 7 21:30:38.131118811 (11h50m59.875175683s) [upload=12256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:30:38.131125270 (11h50m59.875182135s) [upload=12256]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:30:38.131131701 (11h50m59.875188569s) [upload=12256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:31:07.857077892 (11h51m29.601134744s) [upload=12256]: Done uploading segments... Aug 7 21:31:07.857100740 (11h51m29.601157585s) [upload=12256]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.00117736s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [...................................X..XXXXXXXXXXXXXXXXXXXXXX] duration: 30.000540561s FAIL (2025-08-07 21:31:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000568197s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.013589886s FAIL (2025-08-07 21:31:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 7 21:21:33.493714811 (11h41m55.237771657s) [upload=12125]: Upload finished. err: <nil> Aug 7 21:22:47.998800022 (11h43m9.742856869s) [upload=12141]: Starting upload Aug 7 21:22:47.998849424 (11h43m9.742906283s) [upload=12141]: Done starting upload Aug 7 21:22:47.998919383 (11h43m9.742976233s) [upload=12141]: Uploading segments... Aug 7 21:22:47.999001069 (11h43m9.743057938s) [upload=12141]: Got next segment. Inline: true Aug 7 21:22:47.999007866 (11h43m9.743064718s) [upload=12141]: Waiting for error group managing segments... Aug 7 21:22:47.999066684 (11h43m9.743123533s) [upload=12141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:47.999073371 (11h43m9.743130221s) [upload=12141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:47.999077096 (11h43m9.743133947s) [upload=12141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:48.164602559 (11h43m9.90865943s) [upload=12141]: Done uploading segments... Aug 7 21:22:48.164642998 (11h43m9.908699853s) [upload=12141]: Upload finished. err: <nil> Aug 7 21:23:35.990065388 (11h43m57.734122239s) [upload=12154]: Starting upload Aug 7 21:23:35.990143300 (11h43m57.734200146s) [upload=12154]: Done starting upload Aug 7 21:23:35.990217262 (11h43m57.734274108s) [upload=12154]: Uploading segments... Aug 7 21:23:35.990277036 (11h43m57.734333925s) [upload=12154]: Got next segment. Inline: true Aug 7 21:23:35.990283708 (11h43m57.734340569s) [upload=12154]: Waiting for error group managing segments... Aug 7 21:23:35.990337388 (11h43m57.734394253s) [upload=12154]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:35.990343849 (11h43m57.734400714s) [upload=12154]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:35.990347914 (11h43m57.734405001s) [upload=12154]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.150463692 (11h43m57.894520544s) [upload=12154]: Done uploading segments... Aug 7 21:23:36.150483857 (11h43m57.894540707s) [upload=12154]: Upload finished. err: <nil> Aug 7 21:24:13.830997960 (11h44m35.57505482s) [upload=12160]: Starting upload Aug 7 21:24:13.831049433 (11h44m35.575106295s) [upload=12160]: Done starting upload Aug 7 21:24:13.831151824 (11h44m35.575208678s) [upload=12160]: Uploading segments... Aug 7 21:24:13.831192510 (11h44m35.575249377s) [upload=12160]: Got next segment. Inline: true Aug 7 21:24:13.831197614 (11h44m35.575254475s) [upload=12160]: Waiting for error group managing segments... Aug 7 21:24:13.831253497 (11h44m35.57531036s) [upload=12160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:13.831259191 (11h44m35.575316046s) [upload=12160]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:13.831262688 (11h44m35.575319551s) [upload=12160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:13.982951222 (11h44m35.727008092s) [upload=12160]: Done uploading segments... Aug 7 21:24:13.982975878 (11h44m35.727032727s) [upload=12160]: Upload finished. err: <nil> Aug 7 21:24:48.198960455 (11h45m9.943017328s) [upload=12172]: Starting upload Aug 7 21:24:48.199050501 (11h45m9.943107348s) [upload=12172]: Done starting upload Aug 7 21:24:48.199166478 (11h45m9.943223341s) [upload=12172]: Uploading segments... Aug 7 21:24:48.199284289 (11h45m9.943341159s) [upload=12172]: Got next segment. Inline: true Aug 7 21:24:48.199309162 (11h45m9.943366036s) [upload=12172]: Waiting for error group managing segments... Aug 7 21:24:48.199360469 (11h45m9.943417336s) [upload=12172]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:48.199382302 (11h45m9.943439163s) [upload=12172]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:48.199386864 (11h45m9.943443734s) [upload=12172]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:48.353366598 (11h45m10.09742347s) [upload=12172]: Done uploading segments... Aug 7 21:24:48.353410465 (11h45m10.097467327s) [upload=12172]: Upload finished. err: <nil> Aug 7 21:25:56.601363668 (11h46m18.345420518s) [upload=12190]: Starting upload Aug 7 21:25:56.601416202 (11h46m18.345473063s) [upload=12190]: Done starting upload Aug 7 21:25:56.601478987 (11h46m18.345535837s) [upload=12190]: Uploading segments... Aug 7 21:25:56.601519567 (11h46m18.345576436s) [upload=12190]: Got next segment. Inline: true Aug 7 21:25:56.601524613 (11h46m18.345581463s) [upload=12190]: Waiting for error group managing segments... Aug 7 21:25:56.601564530 (11h46m18.345621381s) [upload=12190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:56.601571562 (11h46m18.345628412s) [upload=12190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:56.601574952 (11h46m18.345631806s) [upload=12190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:57.289489478 (11h46m19.033546339s) [upload=12190]: Done uploading segments... Aug 7 21:25:57.289514011 (11h46m19.033570861s) [upload=12190]: Upload finished. err: <nil> Aug 7 21:26:32.814119549 (11h46m54.558176399s) [upload=12198]: Starting upload Aug 7 21:26:32.814178314 (11h46m54.558235164s) [upload=12198]: Done starting upload Aug 7 21:26:32.814230962 (11h46m54.558287814s) [upload=12198]: Uploading segments... Aug 7 21:26:32.814314990 (11h46m54.558371858s) [upload=12198]: Got next segment. Inline: true Aug 7 21:26:32.814321493 (11h46m54.558378353s) [upload=12198]: Waiting for error group managing segments... Aug 7 21:26:32.814387101 (11h46m54.558443951s) [upload=12198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:32.814393685 (11h46m54.558450545s) [upload=12198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:32.814397318 (11h46m54.558454179s) [upload=12198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:32.972796167 (11h46m54.716853027s) [upload=12198]: Done uploading segments... Aug 7 21:26:32.972815783 (11h46m54.716872632s) [upload=12198]: Upload finished. err: <nil> Aug 7 21:27:41.280232963 (11h48m3.024289826s) [upload=12214]: Starting upload Aug 7 21:27:41.280291948 (11h48m3.024348811s) [upload=12214]: Done starting upload Aug 7 21:27:41.280410716 (11h48m3.024467565s) [upload=12214]: Uploading segments... Aug 7 21:27:41.280478626 (11h48m3.024535487s) [upload=12214]: Got next segment. Inline: true Aug 7 21:27:41.280485326 (11h48m3.024542193s) [upload=12214]: Waiting for error group managing segments... Aug 7 21:27:41.280533304 (11h48m3.024590156s) [upload=12214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:41.280538628 (11h48m3.024595488s) [upload=12214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:41.280542274 (11h48m3.02459914s) [upload=12214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:41.433978896 (11h48m3.178035773s) [upload=12214]: Done uploading segments... Aug 7 21:27:41.434003733 (11h48m3.178060594s) [upload=12214]: Upload finished. err: <nil> Aug 7 21:28:53.851663561 (11h49m15.595720412s) [upload=12231]: Starting upload Aug 7 21:28:53.851736366 (11h49m15.595793226s) [upload=12231]: Done starting upload Aug 7 21:28:53.851804427 (11h49m15.595861277s) [upload=12231]: Uploading segments... Aug 7 21:28:53.851851609 (11h49m15.595908471s) [upload=12231]: Got next segment. Inline: true Aug 7 21:28:53.851857251 (11h49m15.595914111s) [upload=12231]: Waiting for error group managing segments... Aug 7 21:28:53.851899816 (11h49m15.595956668s) [upload=12231]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:53.851905032 (11h49m15.595961892s) [upload=12231]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:53.851908856 (11h49m15.595965718s) [upload=12231]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:54.191384948 (11h49m15.935441808s) [upload=12231]: Done uploading segments... Aug 7 21:28:54.191405192 (11h49m15.935462045s) [upload=12231]: Upload finished. err: <nil> Aug 7 21:29:19.999054333 (11h49m41.74311118s) [upload=12237]: Starting upload Aug 7 21:29:19.999128859 (11h49m41.743185722s) [upload=12237]: Done starting upload Aug 7 21:29:19.999188080 (11h49m41.743244931s) [upload=12237]: Uploading segments... Aug 7 21:29:19.999327277 (11h49m41.74338415s) [upload=12237]: Got next segment. Inline: true Aug 7 21:29:19.999335397 (11h49m41.743392265s) [upload=12237]: Waiting for error group managing segments... Aug 7 21:29:19.999418844 (11h49m41.743475691s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:19.999426848 (11h49m41.743483712s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:19.999430264 (11h49m41.743487128s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:21.165098561 (11h49m42.909155421s) [upload=12237]: Done uploading segments... Aug 7 21:29:21.165118654 (11h49m42.909175505s) [upload=12237]: Upload finished. err: <nil> Aug 7 21:29:59.841735510 (11h50m21.585792373s) [upload=12245]: Starting upload Aug 7 21:29:59.841811008 (11h50m21.585867874s) [upload=12245]: Done starting upload Aug 7 21:29:59.841889062 (11h50m21.585945909s) [upload=12245]: Uploading segments... Aug 7 21:29:59.841961319 (11h50m21.58601819s) [upload=12245]: Got next segment. Inline: true Aug 7 21:29:59.841966490 (11h50m21.586023336s) [upload=12245]: Waiting for error group managing segments... Aug 7 21:29:59.842009456 (11h50m21.586066303s) [upload=12245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:59.842013678 (11h50m21.586070524s) [upload=12245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:59.842016402 (11h50m21.586073249s) [upload=12245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:59.937966407 (11h50m21.682023257s) [upload=12245]: Done uploading segments... Aug 7 21:29:59.937989326 (11h50m21.682046187s) [upload=12245]: Upload finished. err: metaclient: Too Many Requests Duration: 1.013624512s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 282.355911ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:30:52 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 282.404609ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 22 out of 60 is failed (threshold=3) [......................................XXXXXXXXXXXXXXXXXXXXXX] duration: 30.000660123s FAIL (2025-08-07 21:30:43 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.00067791s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 16 out of 60 is failed (threshold=3) [............................................XXXXXXXXXXXXXXXX] duration: 30.000703741s FAIL (2025-08-07 21:30:41 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000730669s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 20 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXXXXXXXXXXX] duration: 30.001228105s FAIL (2025-08-07 21:30:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001266217s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 23 out of 60 is failed (threshold=3) [....................................XX.XXXXXXXXXXXXXXXXXXXXX] duration: 30.000610672s FAIL (2025-08-07 21:30:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 42m25.662892995s) [upload=12132]: Uploading segments... Aug 7 21:22:03.918882565 (11h42m25.662939434s) [upload=12132]: Got next segment. Inline: true Aug 7 21:22:03.918887612 (11h42m25.662944465s) [upload=12132]: Waiting for error group managing segments... Aug 7 21:22:03.918934357 (11h42m25.662991223s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:03.918943101 (11h42m25.662999966s) [upload=12132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:03.918952300 (11h42m25.663009183s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:33.649567967 (11h42m55.393624817s) [upload=12132]: Done uploading segments... Aug 7 21:22:33.649593937 (11h42m55.3936508s) [upload=12132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:06.900544850 (11h43m28.644601713s) [upload=12145]: Starting upload Aug 7 21:23:06.900612638 (11h43m28.644669489s) [upload=12145]: Done starting upload Aug 7 21:23:06.901077845 (11h43m28.645134714s) [upload=12145]: Uploading segments... Aug 7 21:23:06.901186098 (11h43m28.645242968s) [upload=12145]: Got next segment. Inline: true Aug 7 21:23:06.901209675 (11h43m28.645266536s) [upload=12145]: Waiting for error group managing segments... Aug 7 21:23:06.901265651 (11h43m28.645322518s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:06.901287666 (11h43m28.645344528s) [upload=12145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:06.901291297 (11h43m28.645348161s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.622244822 (11h43m58.366301669s) [upload=12145]: Done uploading segments... Aug 7 21:23:36.622267267 (11h43m58.366324114s) [upload=12145]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:20.942800102 (11h44m42.686856953s) [upload=12164]: Starting upload Aug 7 21:24:20.942876168 (11h44m42.686933032s) [upload=12164]: Done starting upload Aug 7 21:24:20.942953347 (11h44m42.687010195s) [upload=12164]: Uploading segments... Aug 7 21:24:20.943004925 (11h44m42.687061797s) [upload=12164]: Got next segment. Inline: true Aug 7 21:24:20.943010718 (11h44m42.687067582s) [upload=12164]: Waiting for error group managing segments... Aug 7 21:24:20.943057728 (11h44m42.687114586s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:20.943063424 (11h44m42.687120287s) [upload=12164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:20.943066967 (11h44m42.687123828s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:50.669999700 (11h45m12.414056561s) [upload=12164]: Done uploading segments... Aug 7 21:24:50.670022022 (11h45m12.414078882s) [upload=12164]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:15.759998843 (11h45m37.504055689s) [upload=12180]: Starting upload Aug 7 21:25:15.760060918 (11h45m37.504117764s) [upload=12180]: Done starting upload Aug 7 21:25:15.760121258 (11h45m37.504178104s) [upload=12180]: Uploading segments... Aug 7 21:25:15.760159734 (11h45m37.504216604s) [upload=12180]: Got next segment. Inline: true Aug 7 21:25:15.760164169 (11h45m37.504221029s) [upload=12180]: Waiting for error group managing segments... Aug 7 21:25:15.760202724 (11h45m37.504259584s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:15.760207148 (11h45m37.504264007s) [upload=12180]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:15.760212108 (11h45m37.504268971s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:45.477193191 (11h46m7.22125006s) [upload=12180]: Done uploading segments... Aug 7 21:25:45.477222897 (11h46m7.22127976s) [upload=12180]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:45.889252639 (11h46m7.633309501s) [upload=12186]: Starting upload Aug 7 21:25:45.889337905 (11h46m7.633394765s) [upload=12186]: Done starting upload Aug 7 21:25:45.889442879 (11h46m7.633499729s) [upload=12186]: Uploading segments... Aug 7 21:25:45.889501484 (11h46m7.633558347s) [upload=12186]: Got next segment. Inline: true Aug 7 21:25:45.889507484 (11h46m7.633564344s) [upload=12186]: Waiting for error group managing segments... Aug 7 21:25:45.889551118 (11h46m7.633607968s) [upload=12186]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:45.889556440 (11h46m7.633613289s) [upload=12186]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:45.889559845 (11h46m7.633616705s) [upload=12186]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:15.619717489 (11h46m37.363774339s) [upload=12186]: Done uploading segments... Aug 7 21:26:15.619739739 (11h46m37.363796589s) [upload=12186]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:40.252456263 (11h47m1.996513132s) [upload=12201]: Starting upload Aug 7 21:26:40.252598168 (11h47m1.996655038s) [upload=12201]: Done starting upload Aug 7 21:26:40.252702071 (11h47m1.996758923s) [upload=12201]: Uploading segments... Aug 7 21:26:40.252782136 (11h47m1.996839003s) [upload=12201]: Got next segment. Inline: true Aug 7 21:26:40.252790092 (11h47m1.996846954s) [upload=12201]: Waiting for error group managing segments... Aug 7 21:26:40.252842463 (11h47m1.996899315s) [upload=12201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:40.252849740 (11h47m1.996906592s) [upload=12201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:40.252854153 (11h47m1.996911015s) [upload=12201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:09.981768433 (11h47m31.725825298s) [upload=12201]: Done uploading segments... Aug 7 21:27:09.981793631 (11h47m31.725850483s) [upload=12201]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:26.837217755 (11h47m48.581274619s) [upload=12211]: Starting upload Aug 7 21:27:26.837300223 (11h47m48.581357084s) [upload=12211]: Done starting upload Aug 7 21:27:26.837385984 (11h47m48.581442831s) [upload=12211]: Uploading segments... Aug 7 21:27:26.837444443 (11h47m48.581501313s) [upload=12211]: Got next segment. Inline: true Aug 7 21:27:26.837449770 (11h47m48.581506619s) [upload=12211]: Waiting for error group managing segments... Aug 7 21:27:26.837517257 (11h47m48.581574107s) [upload=12211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:26.837523154 (11h47m48.581580003s) [upload=12211]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:26.837527097 (11h47m48.58158395s) [upload=12211]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:56.564982463 (11h48m18.309039313s) [upload=12211]: Done uploading segments... Aug 7 21:27:56.565004020 (11h48m18.309060871s) [upload=12211]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:36.367399060 (11h48m58.11145591s) [upload=12226]: Starting upload Aug 7 21:28:36.367588554 (11h48m58.111645417s) [upload=12226]: Done starting upload Aug 7 21:28:36.367664912 (11h48m58.111721765s) [upload=12226]: Uploading segments... Aug 7 21:28:36.367723895 (11h48m58.111780757s) [upload=12226]: Got next segment. Inline: true Aug 7 21:28:36.367729331 (11h48m58.111786192s) [upload=12226]: Waiting for error group managing segments... Aug 7 21:28:36.367785894 (11h48m58.111842754s) [upload=12226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:36.367791010 (11h48m58.11184787s) [upload=12226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:36.367794492 (11h48m58.111851352s) [upload=12226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:06.092890267 (11h49m27.836947129s) [upload=12226]: Done uploading segments... Aug 7 21:29:06.092932205 (11h49m27.836989051s) [upload=12226]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:29:06.522310183 (11h49m28.266367033s) [upload=12233]: Starting upload Aug 7 21:29:06.522388772 (11h49m28.266445633s) [upload=12233]: Done starting upload Aug 7 21:29:06.522479144 (11h49m28.266535995s) [upload=12233]: Uploading segments... Aug 7 21:29:06.522560400 (11h49m28.266617269s) [upload=12233]: Got next segment. Inline: true Aug 7 21:29:06.522566995 (11h49m28.266623844s) [upload=12233]: Waiting for error group managing segments... Aug 7 21:29:06.522615251 (11h49m28.266672101s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:06.522621757 (11h49m28.266678608s) [upload=12233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:06.522630028 (11h49m28.266686879s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:36.250046349 (11h49m57.994103211s) [upload=12233]: Done uploading segments... Aug 7 21:29:36.250088941 (11h49m57.994145808s) [upload=12233]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:30:03.947527959 (11h50m25.691584822s) [upload=12249]: Starting upload Aug 7 21:30:03.947767439 (11h50m25.691824311s) [upload=12249]: Done starting upload Aug 7 21:30:03.959515214 (11h50m25.703572066s) [upload=12249]: Uploading segments... Aug 7 21:30:03.959659046 (11h50m25.703715962s) [upload=12249]: Got next segment. Inline: true Aug 7 21:30:03.959668426 (11h50m25.703725302s) [upload=12249]: Waiting for error group managing segments... Aug 7 21:30:03.959783210 (11h50m25.703840082s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:30:03.959790677 (11h50m25.703847531s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:30:03.959795320 (11h50m25.703852169s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:30:33.655073335 (11h50m55.399130195s) [upload=12249]: Done uploading segments... Aug 7 21:30:33.655096560 (11h50m55.399153412s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000657557s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 16 out of 60 is failed (threshold=3) [............................................XXXXXXXXXXXXXXXX] duration: 1m0.000665321s FAIL (2025-08-07 21:30:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00068694s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 15 out of 60 is failed (threshold=3) [............................................X.XXXXXXXXXXXXXX] duration: 1m0.000749457s FAIL (2025-08-07 21:30:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ) [upload=12103]: Uploading segments... Aug 7 21:20:04.793331764 (11h40m26.537388638s) [upload=12103]: Got next segment. Inline: true Aug 7 21:20:04.793339748 (11h40m26.537396613s) [upload=12103]: Waiting for error group managing segments... Aug 7 21:20:04.793417981 (11h40m26.537474831s) [upload=12103]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:04.793424412 (11h40m26.537481262s) [upload=12103]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:04.793427866 (11h40m26.537484716s) [upload=12103]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:04.512178875 (11h41m26.256235737s) [upload=12103]: Done uploading segments... Aug 7 21:21:04.512205338 (11h41m26.256262189s) [upload=12103]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:14.239199000 (11h41m35.983255849s) [upload=12119]: Starting upload Aug 7 21:21:14.239282320 (11h41m35.983339189s) [upload=12119]: Done starting upload Aug 7 21:21:14.239356308 (11h41m35.98341316s) [upload=12119]: Uploading segments... Aug 7 21:21:14.239528709 (11h41m35.983585615s) [upload=12119]: Got next segment. Inline: true Aug 7 21:21:14.239538735 (11h41m35.983595603s) [upload=12119]: Waiting for error group managing segments... Aug 7 21:21:14.239621427 (11h41m35.983678288s) [upload=12119]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:14.239629030 (11h41m35.983685885s) [upload=12119]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:14.239632488 (11h41m35.98368934s) [upload=12119]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:13.962302553 (11h42m35.706359403s) [upload=12119]: Done uploading segments... Aug 7 21:22:13.962327972 (11h42m35.706384822s) [upload=12119]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:14.360206935 (11h42m36.104263785s) [upload=12134]: Starting upload Aug 7 21:22:14.360271125 (11h42m36.104327976s) [upload=12134]: Done starting upload Aug 7 21:22:14.360336747 (11h42m36.104393607s) [upload=12134]: Uploading segments... Aug 7 21:22:14.360380199 (11h42m36.104437068s) [upload=12134]: Got next segment. Inline: true Aug 7 21:22:14.360385659 (11h42m36.10444252s) [upload=12134]: Waiting for error group managing segments... Aug 7 21:22:14.360431265 (11h42m36.104488116s) [upload=12134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:14.360437033 (11h42m36.104493884s) [upload=12134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:14.360440534 (11h42m36.104497395s) [upload=12134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:14.088328715 (11h43m35.832385565s) [upload=12134]: Done uploading segments... Aug 7 21:23:14.088350462 (11h43m35.832407323s) [upload=12134]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:14.505772185 (11h43m36.249829051s) [upload=12148]: Starting upload Aug 7 21:23:14.505877531 (11h43m36.249934403s) [upload=12148]: Done starting upload Aug 7 21:23:14.505973975 (11h43m36.250030823s) [upload=12148]: Uploading segments... Aug 7 21:23:14.506132768 (11h43m36.250189638s) [upload=12148]: Got next segment. Inline: true Aug 7 21:23:14.506140538 (11h43m36.250197403s) [upload=12148]: Waiting for error group managing segments... Aug 7 21:23:14.506211738 (11h43m36.250268603s) [upload=12148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:14.506218667 (11h43m36.250275531s) [upload=12148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:14.506222979 (11h43m36.250279842s) [upload=12148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:14.230441064 (11h44m35.974497931s) [upload=12148]: Done uploading segments... Aug 7 21:24:14.230468679 (11h44m35.974525551s) [upload=12148]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:14.664138591 (11h44m36.408195463s) [upload=12162]: Starting upload Aug 7 21:24:14.664235374 (11h44m36.408292237s) [upload=12162]: Done starting upload Aug 7 21:24:14.664324426 (11h44m36.408381273s) [upload=12162]: Uploading segments... Aug 7 21:24:14.664378438 (11h44m36.408435308s) [upload=12162]: Got next segment. Inline: true Aug 7 21:24:14.664385109 (11h44m36.408441977s) [upload=12162]: Waiting for error group managing segments... Aug 7 21:24:14.664444486 (11h44m36.408501361s) [upload=12162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:14.664452219 (11h44m36.408509087s) [upload=12162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:14.664455983 (11h44m36.40851285s) [upload=12162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:14.390008254 (11h45m36.134065107s) [upload=12162]: Done uploading segments... Aug 7 21:25:14.390037098 (11h45m36.134093947s) [upload=12162]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:14.891337243 (11h45m36.635394094s) [upload=12178]: Starting upload Aug 7 21:25:14.891405973 (11h45m36.635462834s) [upload=12178]: Done starting upload Aug 7 21:25:14.891587260 (11h45m36.635644108s) [upload=12178]: Uploading segments... Aug 7 21:25:14.891659588 (11h45m36.635716456s) [upload=12178]: Got next segment. Inline: true Aug 7 21:25:14.891666181 (11h45m36.635723042s) [upload=12178]: Waiting for error group managing segments... Aug 7 21:25:14.891713596 (11h45m36.635770446s) [upload=12178]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:14.891719665 (11h45m36.635776525s) [upload=12178]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:14.891723701 (11h45m36.635780562s) [upload=12178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:14.618805682 (11h46m36.362862552s) [upload=12178]: Done uploading segments... Aug 7 21:26:14.618848987 (11h46m36.36290586s) [upload=12178]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:15.042634192 (11h46m36.786691042s) [upload=12195]: Starting upload Aug 7 21:26:15.042707477 (11h46m36.786764348s) [upload=12195]: Done starting upload Aug 7 21:26:15.042793174 (11h46m36.786850024s) [upload=12195]: Uploading segments... Aug 7 21:26:15.042848386 (11h46m36.78690526s) [upload=12195]: Got next segment. Inline: true Aug 7 21:26:15.042855793 (11h46m36.786912664s) [upload=12195]: Waiting for error group managing segments... Aug 7 21:26:15.042911164 (11h46m36.786968025s) [upload=12195]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:15.042918440 (11h46m36.786975312s) [upload=12195]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:15.042924061 (11h46m36.78698093s) [upload=12195]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:14.771724725 (11h47m36.515781576s) [upload=12195]: Done uploading segments... Aug 7 21:27:14.771758030 (11h47m36.515814885s) [upload=12195]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:15.202352221 (11h47m36.946409081s) [upload=12209]: Starting upload Aug 7 21:27:15.202440356 (11h47m36.946497217s) [upload=12209]: Done starting upload Aug 7 21:27:15.202525888 (11h47m36.946582739s) [upload=12209]: Uploading segments... Aug 7 21:27:15.202588330 (11h47m36.946645196s) [upload=12209]: Got next segment. Inline: true Aug 7 21:27:15.202594295 (11h47m36.946651156s) [upload=12209]: Waiting for error group managing segments... Aug 7 21:27:15.202651093 (11h47m36.946707944s) [upload=12209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:15.202656763 (11h47m36.946713624s) [upload=12209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:15.202660199 (11h47m36.946717059s) [upload=12209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:14.931851536 (11h48m36.675908396s) [upload=12209]: Done uploading segments... Aug 7 21:28:14.931874519 (11h48m36.67593138s) [upload=12209]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:21.072819157 (11h48m42.816876017s) [upload=12223]: Starting upload Aug 7 21:28:21.072899208 (11h48m42.816956068s) [upload=12223]: Done starting upload Aug 7 21:28:21.072981263 (11h48m42.817038113s) [upload=12223]: Uploading segments... Aug 7 21:28:21.073042235 (11h48m42.817099103s) [upload=12223]: Got next segment. Inline: true Aug 7 21:28:21.073047353 (11h48m42.817104214s) [upload=12223]: Waiting for error group managing segments... Aug 7 21:28:21.073104067 (11h48m42.817160927s) [upload=12223]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:21.073109154 (11h48m42.817166016s) [upload=12223]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:21.073112378 (11h48m42.817169238s) [upload=12223]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:20.797200985 (11h49m42.541258063s) [upload=12223]: Done uploading segments... Aug 7 21:29:20.797233820 (11h49m42.541290687s) [upload=12223]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:29:21.230279976 (11h49m42.974336838s) [upload=12238]: Starting upload Aug 7 21:29:21.230391607 (11h49m42.974448467s) [upload=12238]: Done starting upload Aug 7 21:29:21.230482429 (11h49m42.974539278s) [upload=12238]: Uploading segments... Aug 7 21:29:21.230606006 (11h49m42.974662867s) [upload=12238]: Got next segment. Inline: true Aug 7 21:29:21.230613609 (11h49m42.974670469s) [upload=12238]: Waiting for error group managing segments... Aug 7 21:29:21.230660759 (11h49m42.974717609s) [upload=12238]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:21.230666510 (11h49m42.974723374s) [upload=12238]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:21.230691185 (11h49m42.974748046s) [upload=12238]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:30:20.959347392 (11h50m42.703404243s) [upload=12238]: Done uploading segments... Aug 7 21:30:20.959370210 (11h50m42.70342706s) [upload=12238]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.000769707s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [.......................................XXXXXXXXXXXXXXXXXXXXX] duration: 30.001348426s FAIL (2025-08-07 21:30:13 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001372039s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [....................................X..XXXXXXXXXXXXXXXXXXXXX] duration: 30.001483366s FAIL (2025-08-07 21:30:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001506774s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X......................X.....................] duration: 2.707440811s FAIL (2025-08-07 21:30:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== oad finished. err: <nil> Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments... Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment... Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment. Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments... Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces. Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments... Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil> Duration: 2.707467832s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X..........................................................X] duration: 377.806291ms FAIL (2025-08-07 21:29:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 7 21:21:33.493714811 (11h41m55.237771657s) [upload=12125]: Upload finished. err: <nil> Aug 7 21:22:47.998800022 (11h43m9.742856869s) [upload=12141]: Starting upload Aug 7 21:22:47.998849424 (11h43m9.742906283s) [upload=12141]: Done starting upload Aug 7 21:22:47.998919383 (11h43m9.742976233s) [upload=12141]: Uploading segments... Aug 7 21:22:47.999001069 (11h43m9.743057938s) [upload=12141]: Got next segment. Inline: true Aug 7 21:22:47.999007866 (11h43m9.743064718s) [upload=12141]: Waiting for error group managing segments... Aug 7 21:22:47.999066684 (11h43m9.743123533s) [upload=12141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:47.999073371 (11h43m9.743130221s) [upload=12141]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:47.999077096 (11h43m9.743133947s) [upload=12141]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:48.164602559 (11h43m9.90865943s) [upload=12141]: Done uploading segments... Aug 7 21:22:48.164642998 (11h43m9.908699853s) [upload=12141]: Upload finished. err: <nil> Aug 7 21:23:35.990065388 (11h43m57.734122239s) [upload=12154]: Starting upload Aug 7 21:23:35.990143300 (11h43m57.734200146s) [upload=12154]: Done starting upload Aug 7 21:23:35.990217262 (11h43m57.734274108s) [upload=12154]: Uploading segments... Aug 7 21:23:35.990277036 (11h43m57.734333925s) [upload=12154]: Got next segment. Inline: true Aug 7 21:23:35.990283708 (11h43m57.734340569s) [upload=12154]: Waiting for error group managing segments... Aug 7 21:23:35.990337388 (11h43m57.734394253s) [upload=12154]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:35.990343849 (11h43m57.734400714s) [upload=12154]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:35.990347914 (11h43m57.734405001s) [upload=12154]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.150463692 (11h43m57.894520544s) [upload=12154]: Done uploading segments... Aug 7 21:23:36.150483857 (11h43m57.894540707s) [upload=12154]: Upload finished. err: <nil> Aug 7 21:24:13.830997960 (11h44m35.57505482s) [upload=12160]: Starting upload Aug 7 21:24:13.831049433 (11h44m35.575106295s) [upload=12160]: Done starting upload Aug 7 21:24:13.831151824 (11h44m35.575208678s) [upload=12160]: Uploading segments... Aug 7 21:24:13.831192510 (11h44m35.575249377s) [upload=12160]: Got next segment. Inline: true Aug 7 21:24:13.831197614 (11h44m35.575254475s) [upload=12160]: Waiting for error group managing segments... Aug 7 21:24:13.831253497 (11h44m35.57531036s) [upload=12160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:13.831259191 (11h44m35.575316046s) [upload=12160]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:13.831262688 (11h44m35.575319551s) [upload=12160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:13.982951222 (11h44m35.727008092s) [upload=12160]: Done uploading segments... Aug 7 21:24:13.982975878 (11h44m35.727032727s) [upload=12160]: Upload finished. err: <nil> Aug 7 21:24:48.198960455 (11h45m9.943017328s) [upload=12172]: Starting upload Aug 7 21:24:48.199050501 (11h45m9.943107348s) [upload=12172]: Done starting upload Aug 7 21:24:48.199166478 (11h45m9.943223341s) [upload=12172]: Uploading segments... Aug 7 21:24:48.199284289 (11h45m9.943341159s) [upload=12172]: Got next segment. Inline: true Aug 7 21:24:48.199309162 (11h45m9.943366036s) [upload=12172]: Waiting for error group managing segments... Aug 7 21:24:48.199360469 (11h45m9.943417336s) [upload=12172]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:48.199382302 (11h45m9.943439163s) [upload=12172]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:48.199386864 (11h45m9.943443734s) [upload=12172]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:48.353366598 (11h45m10.09742347s) [upload=12172]: Done uploading segments... Aug 7 21:24:48.353410465 (11h45m10.097467327s) [upload=12172]: Upload finished. err: <nil> Aug 7 21:25:56.601363668 (11h46m18.345420518s) [upload=12190]: Starting upload Aug 7 21:25:56.601416202 (11h46m18.345473063s) [upload=12190]: Done starting upload Aug 7 21:25:56.601478987 (11h46m18.345535837s) [upload=12190]: Uploading segments... Aug 7 21:25:56.601519567 (11h46m18.345576436s) [upload=12190]: Got next segment. Inline: true Aug 7 21:25:56.601524613 (11h46m18.345581463s) [upload=12190]: Waiting for error group managing segments... Aug 7 21:25:56.601564530 (11h46m18.345621381s) [upload=12190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:56.601571562 (11h46m18.345628412s) [upload=12190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:56.601574952 (11h46m18.345631806s) [upload=12190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:57.289489478 (11h46m19.033546339s) [upload=12190]: Done uploading segments... Aug 7 21:25:57.289514011 (11h46m19.033570861s) [upload=12190]: Upload finished. err: <nil> Aug 7 21:26:32.814119549 (11h46m54.558176399s) [upload=12198]: Starting upload Aug 7 21:26:32.814178314 (11h46m54.558235164s) [upload=12198]: Done starting upload Aug 7 21:26:32.814230962 (11h46m54.558287814s) [upload=12198]: Uploading segments... Aug 7 21:26:32.814314990 (11h46m54.558371858s) [upload=12198]: Got next segment. Inline: true Aug 7 21:26:32.814321493 (11h46m54.558378353s) [upload=12198]: Waiting for error group managing segments... Aug 7 21:26:32.814387101 (11h46m54.558443951s) [upload=12198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:32.814393685 (11h46m54.558450545s) [upload=12198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:32.814397318 (11h46m54.558454179s) [upload=12198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:32.972796167 (11h46m54.716853027s) [upload=12198]: Done uploading segments... Aug 7 21:26:32.972815783 (11h46m54.716872632s) [upload=12198]: Upload finished. err: <nil> Aug 7 21:27:41.280232963 (11h48m3.024289826s) [upload=12214]: Starting upload Aug 7 21:27:41.280291948 (11h48m3.024348811s) [upload=12214]: Done starting upload Aug 7 21:27:41.280410716 (11h48m3.024467565s) [upload=12214]: Uploading segments... Aug 7 21:27:41.280478626 (11h48m3.024535487s) [upload=12214]: Got next segment. Inline: true Aug 7 21:27:41.280485326 (11h48m3.024542193s) [upload=12214]: Waiting for error group managing segments... Aug 7 21:27:41.280533304 (11h48m3.024590156s) [upload=12214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:41.280538628 (11h48m3.024595488s) [upload=12214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:41.280542274 (11h48m3.02459914s) [upload=12214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:41.433978896 (11h48m3.178035773s) [upload=12214]: Done uploading segments... Aug 7 21:27:41.434003733 (11h48m3.178060594s) [upload=12214]: Upload finished. err: <nil> Aug 7 21:28:53.851663561 (11h49m15.595720412s) [upload=12231]: Starting upload Aug 7 21:28:53.851736366 (11h49m15.595793226s) [upload=12231]: Done starting upload Aug 7 21:28:53.851804427 (11h49m15.595861277s) [upload=12231]: Uploading segments... Aug 7 21:28:53.851851609 (11h49m15.595908471s) [upload=12231]: Got next segment. Inline: true Aug 7 21:28:53.851857251 (11h49m15.595914111s) [upload=12231]: Waiting for error group managing segments... Aug 7 21:28:53.851899816 (11h49m15.595956668s) [upload=12231]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:53.851905032 (11h49m15.595961892s) [upload=12231]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:53.851908856 (11h49m15.595965718s) [upload=12231]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:54.191384948 (11h49m15.935441808s) [upload=12231]: Done uploading segments... Aug 7 21:28:54.191405192 (11h49m15.935462045s) [upload=12231]: Upload finished. err: <nil> Aug 7 21:29:19.999054333 (11h49m41.74311118s) [upload=12237]: Starting upload Aug 7 21:29:19.999128859 (11h49m41.743185722s) [upload=12237]: Done starting upload Aug 7 21:29:19.999188080 (11h49m41.743244931s) [upload=12237]: Uploading segments... Aug 7 21:29:19.999327277 (11h49m41.74338415s) [upload=12237]: Got next segment. Inline: true Aug 7 21:29:19.999335397 (11h49m41.743392265s) [upload=12237]: Waiting for error group managing segments... Aug 7 21:29:19.999418844 (11h49m41.743475691s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:19.999426848 (11h49m41.743483712s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:19.999430264 (11h49m41.743487128s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:21.165098561 (11h49m42.909155421s) [upload=12237]: Done uploading segments... Aug 7 21:29:21.165118654 (11h49m42.909175505s) [upload=12237]: Upload finished. err: <nil> Aug 7 21:29:59.841735510 (11h50m21.585792373s) [upload=12245]: Starting upload Aug 7 21:29:59.841811008 (11h50m21.585867874s) [upload=12245]: Done starting upload Aug 7 21:29:59.841889062 (11h50m21.585945909s) [upload=12245]: Uploading segments... Aug 7 21:29:59.841961319 (11h50m21.58601819s) [upload=12245]: Got next segment. Inline: true Aug 7 21:29:59.841966490 (11h50m21.586023336s) [upload=12245]: Waiting for error group managing segments... Aug 7 21:29:59.842009456 (11h50m21.586066303s) [upload=12245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:59.842013678 (11h50m21.586070524s) [upload=12245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:59.842016402 (11h50m21.586073249s) [upload=12245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:59.937966407 (11h50m21.682023257s) [upload=12245]: Done uploading segments... Aug 7 21:29:59.937989326 (11h50m21.682046187s) [upload=12245]: Upload finished. err: metaclient: Too Many Requests Duration: 377.836471ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration: 30.00086361s FAIL (2025-08-07 21:29:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000880451s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 149.224261ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:29:45 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 149.250525ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXXXXXX] duration: 30.001348764s FAIL (2025-08-07 21:29:44 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001365643s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 22 out of 60 is failed (threshold=3) [.....................................XX.XXXXXXXXXXXXXXXXXXXX] duration: 30.000696238s FAIL (2025-08-07 21:29:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 41m55.527288844s) [upload=12126]: Uploading segments... Aug 7 21:21:33.783290191 (11h41m55.527347062s) [upload=12126]: Got next segment. Inline: true Aug 7 21:21:33.783297437 (11h41m55.527354312s) [upload=12126]: Waiting for error group managing segments... Aug 7 21:21:33.783352192 (11h41m55.527409054s) [upload=12126]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:33.783359317 (11h41m55.527416182s) [upload=12126]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:33.783363849 (11h41m55.527420713s) [upload=12126]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:03.508585017 (11h42m25.252641871s) [upload=12126]: Done uploading segments... Aug 7 21:22:03.508625142 (11h42m25.252681994s) [upload=12126]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:03.918704824 (11h42m25.662761676s) [upload=12132]: Starting upload Aug 7 21:22:03.918771848 (11h42m25.662828712s) [upload=12132]: Done starting upload Aug 7 21:22:03.918836144 (11h42m25.662892995s) [upload=12132]: Uploading segments... Aug 7 21:22:03.918882565 (11h42m25.662939434s) [upload=12132]: Got next segment. Inline: true Aug 7 21:22:03.918887612 (11h42m25.662944465s) [upload=12132]: Waiting for error group managing segments... Aug 7 21:22:03.918934357 (11h42m25.662991223s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:03.918943101 (11h42m25.662999966s) [upload=12132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:03.918952300 (11h42m25.663009183s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:33.649567967 (11h42m55.393624817s) [upload=12132]: Done uploading segments... Aug 7 21:22:33.649593937 (11h42m55.3936508s) [upload=12132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:06.900544850 (11h43m28.644601713s) [upload=12145]: Starting upload Aug 7 21:23:06.900612638 (11h43m28.644669489s) [upload=12145]: Done starting upload Aug 7 21:23:06.901077845 (11h43m28.645134714s) [upload=12145]: Uploading segments... Aug 7 21:23:06.901186098 (11h43m28.645242968s) [upload=12145]: Got next segment. Inline: true Aug 7 21:23:06.901209675 (11h43m28.645266536s) [upload=12145]: Waiting for error group managing segments... Aug 7 21:23:06.901265651 (11h43m28.645322518s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:06.901287666 (11h43m28.645344528s) [upload=12145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:06.901291297 (11h43m28.645348161s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.622244822 (11h43m58.366301669s) [upload=12145]: Done uploading segments... Aug 7 21:23:36.622267267 (11h43m58.366324114s) [upload=12145]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:20.942800102 (11h44m42.686856953s) [upload=12164]: Starting upload Aug 7 21:24:20.942876168 (11h44m42.686933032s) [upload=12164]: Done starting upload Aug 7 21:24:20.942953347 (11h44m42.687010195s) [upload=12164]: Uploading segments... Aug 7 21:24:20.943004925 (11h44m42.687061797s) [upload=12164]: Got next segment. Inline: true Aug 7 21:24:20.943010718 (11h44m42.687067582s) [upload=12164]: Waiting for error group managing segments... Aug 7 21:24:20.943057728 (11h44m42.687114586s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:20.943063424 (11h44m42.687120287s) [upload=12164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:20.943066967 (11h44m42.687123828s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:50.669999700 (11h45m12.414056561s) [upload=12164]: Done uploading segments... Aug 7 21:24:50.670022022 (11h45m12.414078882s) [upload=12164]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:15.759998843 (11h45m37.504055689s) [upload=12180]: Starting upload Aug 7 21:25:15.760060918 (11h45m37.504117764s) [upload=12180]: Done starting upload Aug 7 21:25:15.760121258 (11h45m37.504178104s) [upload=12180]: Uploading segments... Aug 7 21:25:15.760159734 (11h45m37.504216604s) [upload=12180]: Got next segment. Inline: true Aug 7 21:25:15.760164169 (11h45m37.504221029s) [upload=12180]: Waiting for error group managing segments... Aug 7 21:25:15.760202724 (11h45m37.504259584s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:15.760207148 (11h45m37.504264007s) [upload=12180]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:15.760212108 (11h45m37.504268971s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:45.477193191 (11h46m7.22125006s) [upload=12180]: Done uploading segments... Aug 7 21:25:45.477222897 (11h46m7.22127976s) [upload=12180]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:45.889252639 (11h46m7.633309501s) [upload=12186]: Starting upload Aug 7 21:25:45.889337905 (11h46m7.633394765s) [upload=12186]: Done starting upload Aug 7 21:25:45.889442879 (11h46m7.633499729s) [upload=12186]: Uploading segments... Aug 7 21:25:45.889501484 (11h46m7.633558347s) [upload=12186]: Got next segment. Inline: true Aug 7 21:25:45.889507484 (11h46m7.633564344s) [upload=12186]: Waiting for error group managing segments... Aug 7 21:25:45.889551118 (11h46m7.633607968s) [upload=12186]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:45.889556440 (11h46m7.633613289s) [upload=12186]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:45.889559845 (11h46m7.633616705s) [upload=12186]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:15.619717489 (11h46m37.363774339s) [upload=12186]: Done uploading segments... Aug 7 21:26:15.619739739 (11h46m37.363796589s) [upload=12186]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:40.252456263 (11h47m1.996513132s) [upload=12201]: Starting upload Aug 7 21:26:40.252598168 (11h47m1.996655038s) [upload=12201]: Done starting upload Aug 7 21:26:40.252702071 (11h47m1.996758923s) [upload=12201]: Uploading segments... Aug 7 21:26:40.252782136 (11h47m1.996839003s) [upload=12201]: Got next segment. Inline: true Aug 7 21:26:40.252790092 (11h47m1.996846954s) [upload=12201]: Waiting for error group managing segments... Aug 7 21:26:40.252842463 (11h47m1.996899315s) [upload=12201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:40.252849740 (11h47m1.996906592s) [upload=12201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:40.252854153 (11h47m1.996911015s) [upload=12201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:09.981768433 (11h47m31.725825298s) [upload=12201]: Done uploading segments... Aug 7 21:27:09.981793631 (11h47m31.725850483s) [upload=12201]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:26.837217755 (11h47m48.581274619s) [upload=12211]: Starting upload Aug 7 21:27:26.837300223 (11h47m48.581357084s) [upload=12211]: Done starting upload Aug 7 21:27:26.837385984 (11h47m48.581442831s) [upload=12211]: Uploading segments... Aug 7 21:27:26.837444443 (11h47m48.581501313s) [upload=12211]: Got next segment. Inline: true Aug 7 21:27:26.837449770 (11h47m48.581506619s) [upload=12211]: Waiting for error group managing segments... Aug 7 21:27:26.837517257 (11h47m48.581574107s) [upload=12211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:26.837523154 (11h47m48.581580003s) [upload=12211]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:26.837527097 (11h47m48.58158395s) [upload=12211]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:56.564982463 (11h48m18.309039313s) [upload=12211]: Done uploading segments... Aug 7 21:27:56.565004020 (11h48m18.309060871s) [upload=12211]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:36.367399060 (11h48m58.11145591s) [upload=12226]: Starting upload Aug 7 21:28:36.367588554 (11h48m58.111645417s) [upload=12226]: Done starting upload Aug 7 21:28:36.367664912 (11h48m58.111721765s) [upload=12226]: Uploading segments... Aug 7 21:28:36.367723895 (11h48m58.111780757s) [upload=12226]: Got next segment. Inline: true Aug 7 21:28:36.367729331 (11h48m58.111786192s) [upload=12226]: Waiting for error group managing segments... Aug 7 21:28:36.367785894 (11h48m58.111842754s) [upload=12226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:36.367791010 (11h48m58.11184787s) [upload=12226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:36.367794492 (11h48m58.111851352s) [upload=12226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:06.092890267 (11h49m27.836947129s) [upload=12226]: Done uploading segments... Aug 7 21:29:06.092932205 (11h49m27.836989051s) [upload=12226]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:29:06.522310183 (11h49m28.266367033s) [upload=12233]: Starting upload Aug 7 21:29:06.522388772 (11h49m28.266445633s) [upload=12233]: Done starting upload Aug 7 21:29:06.522479144 (11h49m28.266535995s) [upload=12233]: Uploading segments... Aug 7 21:29:06.522560400 (11h49m28.266617269s) [upload=12233]: Got next segment. Inline: true Aug 7 21:29:06.522566995 (11h49m28.266623844s) [upload=12233]: Waiting for error group managing segments... Aug 7 21:29:06.522615251 (11h49m28.266672101s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:06.522621757 (11h49m28.266678608s) [upload=12233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:06.522630028 (11h49m28.266686879s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:36.250046349 (11h49m57.994103211s) [upload=12233]: Done uploading segments... Aug 7 21:29:36.250088941 (11h49m57.994145808s) [upload=12233]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000716191s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X......................X....................] duration: 4.598079016s FAIL (2025-08-07 21:29:25 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== oad finished. err: <nil> Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments... Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment... Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment. Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments... Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces. Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments... Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil> Duration: 4.59810984s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 3.326680629s FAIL (2025-08-07 21:29:23 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== 68895 (10h54m20.003725754s) [upload=11347]: Done uploading segments... Aug 7 20:33:58.259694049 (10h54m20.00375091s) [upload=11347]: Upload finished. err: <nil> Aug 7 20:34:42.291300370 (10h55m4.035357231s) [upload=11357]: Starting upload Aug 7 20:34:42.291392440 (10h55m4.035449315s) [upload=11357]: Done starting upload Aug 7 20:34:42.291505939 (10h55m4.035562788s) [upload=11357]: Uploading segments... Aug 7 20:34:42.291587798 (10h55m4.035644667s) [upload=11357]: Got next segment. Inline: true Aug 7 20:34:42.291610428 (10h55m4.035667289s) [upload=11357]: Waiting for error group managing segments... Aug 7 20:34:42.291660078 (10h55m4.035716927s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:34:42.291665407 (10h55m4.035722257s) [upload=11357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:34:42.291668818 (10h55m4.035725679s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:34:42.445169337 (10h55m4.189226185s) [upload=11357]: Done uploading segments... Aug 7 20:34:42.445187662 (10h55m4.189244508s) [upload=11357]: Upload finished. err: <nil> Aug 7 20:35:56.993979713 (10h56m18.738036563s) [upload=11378]: Starting upload Aug 7 20:35:56.994042734 (10h56m18.738099594s) [upload=11378]: Done starting upload Aug 7 20:35:56.994133564 (10h56m18.738190418s) [upload=11378]: Uploading segments... Aug 7 20:35:56.994188836 (10h56m18.738245705s) [upload=11378]: Got next segment. Inline: true Aug 7 20:35:56.994194253 (10h56m18.738251113s) [upload=11378]: Waiting for error group managing segments... Aug 7 20:35:56.994255059 (10h56m18.738311921s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:35:56.994261721 (10h56m18.738318586s) [upload=11378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:35:56.994266228 (10h56m18.73832309s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:35:57.196915243 (10h56m18.940972094s) [upload=11378]: Done uploading segments... Aug 7 20:35:57.196933580 (10h56m18.940990431s) [upload=11378]: Upload finished. err: <nil> Aug 7 20:36:48.383926585 (10h57m10.127983431s) [upload=11392]: Starting upload Aug 7 20:36:48.383977279 (10h57m10.128034139s) [upload=11392]: Done starting upload Aug 7 20:36:48.384080928 (10h57m10.128137781s) [upload=11392]: Uploading segments... Aug 7 20:36:48.384124913 (10h57m10.128181781s) [upload=11392]: Got next segment. Inline: true Aug 7 20:36:48.384130188 (10h57m10.128187037s) [upload=11392]: Waiting for error group managing segments... Aug 7 20:36:48.384183298 (10h57m10.128240148s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:36:48.384188890 (10h57m10.128245741s) [upload=11392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:36:48.384192353 (10h57m10.128249203s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:36:48.597716681 (10h57m10.341773532s) [upload=11392]: Done uploading segments... Aug 7 20:36:48.597735626 (10h57m10.341792476s) [upload=11392]: Upload finished. err: <nil> Aug 7 20:37:59.886756865 (10h58m21.630813711s) [upload=11411]: Starting upload Aug 7 20:37:59.886810662 (10h58m21.630867522s) [upload=11411]: Done starting upload Aug 7 20:37:59.886870714 (10h58m21.630927561s) [upload=11411]: Uploading segments... Aug 7 20:37:59.886906763 (10h58m21.630963634s) [upload=11411]: Got next segment. Inline: true Aug 7 20:37:59.886911413 (10h58m21.630968259s) [upload=11411]: Waiting for error group managing segments... Aug 7 20:37:59.886948286 (10h58m21.631005132s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:37:59.886952923 (10h58m21.631009769s) [upload=11411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:37:59.886957250 (10h58m21.631014096s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:38:00.073123067 (10h58m21.817179918s) [upload=11411]: Done uploading segments... Aug 7 20:38:00.073142248 (10h58m21.817199097s) [upload=11411]: Upload finished. err: <nil> Aug 7 20:39:06.192304563 (10h59m27.936361423s) [upload=11429]: Starting upload Aug 7 20:39:06.192379945 (10h59m27.936436804s) [upload=11429]: Done starting upload Aug 7 20:39:06.192450925 (10h59m27.936507772s) [upload=11429]: Uploading segments... Aug 7 20:39:06.192493866 (10h59m27.936550726s) [upload=11429]: Got next segment. Inline: true Aug 7 20:39:06.192499400 (10h59m27.93655626s) [upload=11429]: Waiting for error group managing segments... Aug 7 20:39:06.192549058 (10h59m27.936605918s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:06.192553658 (10h59m27.936610518s) [upload=11429]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:06.192556283 (10h59m27.936613143s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:06.410732058 (10h59m28.154788918s) [upload=11429]: Done uploading segments... Aug 7 20:39:06.410751072 (10h59m28.154807921s) [upload=11429]: Upload finished. err: <nil> Aug 7 20:39:38.189437550 (10h59m59.9334944s) [upload=11438]: Starting upload Aug 7 20:39:38.189494699 (10h59m59.933551565s) [upload=11438]: Done starting upload Aug 7 20:39:38.189570885 (10h59m59.933627735s) [upload=11438]: Uploading segments... Aug 7 20:39:38.189613828 (10h59m59.933670698s) [upload=11438]: Got next segment. Inline: true Aug 7 20:39:38.189618898 (10h59m59.933675758s) [upload=11438]: Waiting for error group managing segments... Aug 7 20:39:38.189671547 (10h59m59.933728396s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:38.189676923 (10h59m59.933733774s) [upload=11438]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:38.189680363 (10h59m59.933737214s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:38.339370287 (11h0m0.083427138s) [upload=11438]: Done uploading segments... Aug 7 20:39:38.339394965 (11h0m0.083451822s) [upload=11438]: Upload finished. err: <nil> Aug 7 20:40:13.049368895 (11h0m34.793425745s) [upload=11448]: Starting upload Aug 7 20:40:13.049412890 (11h0m34.793469743s) [upload=11448]: Done starting upload Aug 7 20:40:13.049490173 (11h0m34.793547027s) [upload=11448]: Uploading segments... Aug 7 20:40:13.049531553 (11h0m34.793588423s) [upload=11448]: Got next segment. Inline: true Aug 7 20:40:13.049536305 (11h0m34.79359316s) [upload=11448]: Waiting for error group managing segments... Aug 7 20:40:13.049606089 (11h0m34.793662937s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:40:13.049611863 (11h0m34.793668715s) [upload=11448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:40:13.049615487 (11h0m34.793672338s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:40:13.251485869 (11h0m34.995542721s) [upload=11448]: Done uploading segments... Aug 7 20:40:13.251504252 (11h0m34.995561102s) [upload=11448]: Upload finished. err: <nil> Aug 7 20:41:01.300323709 (11h1m23.044380561s) [upload=11462]: Starting upload Aug 7 20:41:01.300386685 (11h1m23.044443554s) [upload=11462]: Done starting upload Aug 7 20:41:01.300471424 (11h1m23.044528277s) [upload=11462]: Uploading segments... Aug 7 20:41:01.300528867 (11h1m23.044585736s) [upload=11462]: Got next segment. Inline: true Aug 7 20:41:01.300534426 (11h1m23.044591286s) [upload=11462]: Waiting for error group managing segments... Aug 7 20:41:01.300592418 (11h1m23.044649268s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:01.300598189 (11h1m23.04465505s) [upload=11462]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:01.300601616 (11h1m23.044658476s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:01.448083138 (11h1m23.192139989s) [upload=11462]: Done uploading segments... Aug 7 20:41:01.448102791 (11h1m23.192159644s) [upload=11462]: Upload finished. err: <nil> Aug 7 20:41:25.522931110 (11h1m47.266987978s) [upload=11468]: Starting upload Aug 7 20:41:25.523039051 (11h1m47.267095924s) [upload=11468]: Done starting upload Aug 7 20:41:25.523181759 (11h1m47.267238621s) [upload=11468]: Uploading segments... Aug 7 20:41:25.523403073 (11h1m47.267459944s) [upload=11468]: Got next segment. Inline: true Aug 7 20:41:25.523413059 (11h1m47.26746993s) [upload=11468]: Waiting for error group managing segments... Aug 7 20:41:25.523492519 (11h1m47.267549389s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:25.523498700 (11h1m47.267555566s) [upload=11468]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:25.523502761 (11h1m47.267559629s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:25.840530122 (11h1m47.584586983s) [upload=11468]: Done uploading segments... Aug 7 20:41:25.840544406 (11h1m47.584601253s) [upload=11468]: Upload finished. err: <nil> Aug 7 20:41:49.037046771 (11h2m10.781103621s) [upload=11474]: Starting upload Aug 7 20:41:49.037108376 (11h2m10.781165228s) [upload=11474]: Done starting upload Aug 7 20:41:49.037181045 (11h2m10.781237895s) [upload=11474]: Uploading segments... Aug 7 20:41:49.037238734 (11h2m10.781295606s) [upload=11474]: Got next segment. Inline: true Aug 7 20:41:49.037244163 (11h2m10.781301028s) [upload=11474]: Waiting for error group managing segments... Aug 7 20:41:49.037309765 (11h2m10.781366628s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:49.037314702 (11h2m10.781371562s) [upload=11474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:49.037317447 (11h2m10.781374307s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:49.205260584 (11h2m10.949317434s) [upload=11474]: Done uploading segments... Aug 7 20:41:49.205278299 (11h2m10.94933515s) [upload=11474]: Upload finished. err: <nil> Duration: 3.32674267s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXXXXXX] duration: 1m0.001842304s FAIL (2025-08-07 21:29:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001881687s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 20 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXXXXXXXXXXX] duration: 30.001136024s FAIL (2025-08-07 21:29:21 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001160216s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 14 out of 60 is failed (threshold=3) [.............................................X.XXXXXXXXXXXXX] duration: 1m0.001059805s FAIL (2025-08-07 21:29:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ) [upload=12087]: Uploading segments... Aug 7 21:18:55.120142836 (11h39m16.864199705s) [upload=12087]: Got next segment. Inline: true Aug 7 21:18:55.120147776 (11h39m16.864204637s) [upload=12087]: Waiting for error group managing segments... Aug 7 21:18:55.120201712 (11h39m16.864258573s) [upload=12087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:55.120207674 (11h39m16.864264534s) [upload=12087]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:55.120211321 (11h39m16.864268183s) [upload=12087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:54.844368358 (11h40m16.588425206s) [upload=12087]: Done uploading segments... Aug 7 21:19:54.844395070 (11h40m16.588451926s) [upload=12087]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:04.793025148 (11h40m26.537081997s) [upload=12103]: Starting upload Aug 7 21:20:04.793120317 (11h40m26.53717719s) [upload=12103]: Done starting upload Aug 7 21:20:04.793244612 (11h40m26.537301459s) [upload=12103]: Uploading segments... Aug 7 21:20:04.793331764 (11h40m26.537388638s) [upload=12103]: Got next segment. Inline: true Aug 7 21:20:04.793339748 (11h40m26.537396613s) [upload=12103]: Waiting for error group managing segments... Aug 7 21:20:04.793417981 (11h40m26.537474831s) [upload=12103]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:04.793424412 (11h40m26.537481262s) [upload=12103]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:04.793427866 (11h40m26.537484716s) [upload=12103]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:04.512178875 (11h41m26.256235737s) [upload=12103]: Done uploading segments... Aug 7 21:21:04.512205338 (11h41m26.256262189s) [upload=12103]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:14.239199000 (11h41m35.983255849s) [upload=12119]: Starting upload Aug 7 21:21:14.239282320 (11h41m35.983339189s) [upload=12119]: Done starting upload Aug 7 21:21:14.239356308 (11h41m35.98341316s) [upload=12119]: Uploading segments... Aug 7 21:21:14.239528709 (11h41m35.983585615s) [upload=12119]: Got next segment. Inline: true Aug 7 21:21:14.239538735 (11h41m35.983595603s) [upload=12119]: Waiting for error group managing segments... Aug 7 21:21:14.239621427 (11h41m35.983678288s) [upload=12119]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:14.239629030 (11h41m35.983685885s) [upload=12119]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:14.239632488 (11h41m35.98368934s) [upload=12119]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:13.962302553 (11h42m35.706359403s) [upload=12119]: Done uploading segments... Aug 7 21:22:13.962327972 (11h42m35.706384822s) [upload=12119]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:14.360206935 (11h42m36.104263785s) [upload=12134]: Starting upload Aug 7 21:22:14.360271125 (11h42m36.104327976s) [upload=12134]: Done starting upload Aug 7 21:22:14.360336747 (11h42m36.104393607s) [upload=12134]: Uploading segments... Aug 7 21:22:14.360380199 (11h42m36.104437068s) [upload=12134]: Got next segment. Inline: true Aug 7 21:22:14.360385659 (11h42m36.10444252s) [upload=12134]: Waiting for error group managing segments... Aug 7 21:22:14.360431265 (11h42m36.104488116s) [upload=12134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:14.360437033 (11h42m36.104493884s) [upload=12134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:14.360440534 (11h42m36.104497395s) [upload=12134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:14.088328715 (11h43m35.832385565s) [upload=12134]: Done uploading segments... Aug 7 21:23:14.088350462 (11h43m35.832407323s) [upload=12134]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:14.505772185 (11h43m36.249829051s) [upload=12148]: Starting upload Aug 7 21:23:14.505877531 (11h43m36.249934403s) [upload=12148]: Done starting upload Aug 7 21:23:14.505973975 (11h43m36.250030823s) [upload=12148]: Uploading segments... Aug 7 21:23:14.506132768 (11h43m36.250189638s) [upload=12148]: Got next segment. Inline: true Aug 7 21:23:14.506140538 (11h43m36.250197403s) [upload=12148]: Waiting for error group managing segments... Aug 7 21:23:14.506211738 (11h43m36.250268603s) [upload=12148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:14.506218667 (11h43m36.250275531s) [upload=12148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:14.506222979 (11h43m36.250279842s) [upload=12148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:14.230441064 (11h44m35.974497931s) [upload=12148]: Done uploading segments... Aug 7 21:24:14.230468679 (11h44m35.974525551s) [upload=12148]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:14.664138591 (11h44m36.408195463s) [upload=12162]: Starting upload Aug 7 21:24:14.664235374 (11h44m36.408292237s) [upload=12162]: Done starting upload Aug 7 21:24:14.664324426 (11h44m36.408381273s) [upload=12162]: Uploading segments... Aug 7 21:24:14.664378438 (11h44m36.408435308s) [upload=12162]: Got next segment. Inline: true Aug 7 21:24:14.664385109 (11h44m36.408441977s) [upload=12162]: Waiting for error group managing segments... Aug 7 21:24:14.664444486 (11h44m36.408501361s) [upload=12162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:14.664452219 (11h44m36.408509087s) [upload=12162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:14.664455983 (11h44m36.40851285s) [upload=12162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:14.390008254 (11h45m36.134065107s) [upload=12162]: Done uploading segments... Aug 7 21:25:14.390037098 (11h45m36.134093947s) [upload=12162]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:14.891337243 (11h45m36.635394094s) [upload=12178]: Starting upload Aug 7 21:25:14.891405973 (11h45m36.635462834s) [upload=12178]: Done starting upload Aug 7 21:25:14.891587260 (11h45m36.635644108s) [upload=12178]: Uploading segments... Aug 7 21:25:14.891659588 (11h45m36.635716456s) [upload=12178]: Got next segment. Inline: true Aug 7 21:25:14.891666181 (11h45m36.635723042s) [upload=12178]: Waiting for error group managing segments... Aug 7 21:25:14.891713596 (11h45m36.635770446s) [upload=12178]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:14.891719665 (11h45m36.635776525s) [upload=12178]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:14.891723701 (11h45m36.635780562s) [upload=12178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:14.618805682 (11h46m36.362862552s) [upload=12178]: Done uploading segments... Aug 7 21:26:14.618848987 (11h46m36.36290586s) [upload=12178]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:15.042634192 (11h46m36.786691042s) [upload=12195]: Starting upload Aug 7 21:26:15.042707477 (11h46m36.786764348s) [upload=12195]: Done starting upload Aug 7 21:26:15.042793174 (11h46m36.786850024s) [upload=12195]: Uploading segments... Aug 7 21:26:15.042848386 (11h46m36.78690526s) [upload=12195]: Got next segment. Inline: true Aug 7 21:26:15.042855793 (11h46m36.786912664s) [upload=12195]: Waiting for error group managing segments... Aug 7 21:26:15.042911164 (11h46m36.786968025s) [upload=12195]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:15.042918440 (11h46m36.786975312s) [upload=12195]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:15.042924061 (11h46m36.78698093s) [upload=12195]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:14.771724725 (11h47m36.515781576s) [upload=12195]: Done uploading segments... Aug 7 21:27:14.771758030 (11h47m36.515814885s) [upload=12195]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:15.202352221 (11h47m36.946409081s) [upload=12209]: Starting upload Aug 7 21:27:15.202440356 (11h47m36.946497217s) [upload=12209]: Done starting upload Aug 7 21:27:15.202525888 (11h47m36.946582739s) [upload=12209]: Uploading segments... Aug 7 21:27:15.202588330 (11h47m36.946645196s) [upload=12209]: Got next segment. Inline: true Aug 7 21:27:15.202594295 (11h47m36.946651156s) [upload=12209]: Waiting for error group managing segments... Aug 7 21:27:15.202651093 (11h47m36.946707944s) [upload=12209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:15.202656763 (11h47m36.946713624s) [upload=12209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:15.202660199 (11h47m36.946717059s) [upload=12209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:14.931851536 (11h48m36.675908396s) [upload=12209]: Done uploading segments... Aug 7 21:28:14.931874519 (11h48m36.67593138s) [upload=12209]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:21.072819157 (11h48m42.816876017s) [upload=12223]: Starting upload Aug 7 21:28:21.072899208 (11h48m42.816956068s) [upload=12223]: Done starting upload Aug 7 21:28:21.072981263 (11h48m42.817038113s) [upload=12223]: Uploading segments... Aug 7 21:28:21.073042235 (11h48m42.817099103s) [upload=12223]: Got next segment. Inline: true Aug 7 21:28:21.073047353 (11h48m42.817104214s) [upload=12223]: Waiting for error group managing segments... Aug 7 21:28:21.073104067 (11h48m42.817160927s) [upload=12223]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:21.073109154 (11h48m42.817166016s) [upload=12223]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:21.073112378 (11h48m42.817169238s) [upload=12223]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:20.797200985 (11h49m42.541258063s) [upload=12223]: Done uploading segments... Aug 7 21:29:20.797233820 (11h49m42.541290687s) [upload=12223]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.001089427s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration: 30.000751653s FAIL (2025-08-07 21:29:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000780019s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [............................................XX..............] duration: 315.572059ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:29:17 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 315.598983ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 30.001415255s FAIL (2025-08-07 21:29:14 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001440312s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 21 out of 60 is failed (threshold=3) [......................................XX.XXXXXXXXXXXXXXXXXXX] duration: 30.000733064s FAIL (2025-08-07 21:29:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h41m25.363265908s) [upload=12116]: Uploading segments... Aug 7 21:21:03.619303607 (11h41m25.363360476s) [upload=12116]: Got next segment. Inline: true Aug 7 21:21:03.619310139 (11h41m25.363366989s) [upload=12116]: Waiting for error group managing segments... Aug 7 21:21:03.619383781 (11h41m25.363440632s) [upload=12116]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:03.619390203 (11h41m25.363447055s) [upload=12116]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:03.619504810 (11h41m25.363561678s) [upload=12116]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:33.340185414 (11h41m55.084242267s) [upload=12116]: Done uploading segments... Aug 7 21:21:33.340212992 (11h41m55.08426986s) [upload=12116]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:33.783052975 (11h41m55.527109844s) [upload=12126]: Starting upload Aug 7 21:21:33.783141523 (11h41m55.527198395s) [upload=12126]: Done starting upload Aug 7 21:21:33.783231997 (11h41m55.527288844s) [upload=12126]: Uploading segments... Aug 7 21:21:33.783290191 (11h41m55.527347062s) [upload=12126]: Got next segment. Inline: true Aug 7 21:21:33.783297437 (11h41m55.527354312s) [upload=12126]: Waiting for error group managing segments... Aug 7 21:21:33.783352192 (11h41m55.527409054s) [upload=12126]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:33.783359317 (11h41m55.527416182s) [upload=12126]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:33.783363849 (11h41m55.527420713s) [upload=12126]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:03.508585017 (11h42m25.252641871s) [upload=12126]: Done uploading segments... Aug 7 21:22:03.508625142 (11h42m25.252681994s) [upload=12126]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:03.918704824 (11h42m25.662761676s) [upload=12132]: Starting upload Aug 7 21:22:03.918771848 (11h42m25.662828712s) [upload=12132]: Done starting upload Aug 7 21:22:03.918836144 (11h42m25.662892995s) [upload=12132]: Uploading segments... Aug 7 21:22:03.918882565 (11h42m25.662939434s) [upload=12132]: Got next segment. Inline: true Aug 7 21:22:03.918887612 (11h42m25.662944465s) [upload=12132]: Waiting for error group managing segments... Aug 7 21:22:03.918934357 (11h42m25.662991223s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:03.918943101 (11h42m25.662999966s) [upload=12132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:03.918952300 (11h42m25.663009183s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:33.649567967 (11h42m55.393624817s) [upload=12132]: Done uploading segments... Aug 7 21:22:33.649593937 (11h42m55.3936508s) [upload=12132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:06.900544850 (11h43m28.644601713s) [upload=12145]: Starting upload Aug 7 21:23:06.900612638 (11h43m28.644669489s) [upload=12145]: Done starting upload Aug 7 21:23:06.901077845 (11h43m28.645134714s) [upload=12145]: Uploading segments... Aug 7 21:23:06.901186098 (11h43m28.645242968s) [upload=12145]: Got next segment. Inline: true Aug 7 21:23:06.901209675 (11h43m28.645266536s) [upload=12145]: Waiting for error group managing segments... Aug 7 21:23:06.901265651 (11h43m28.645322518s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:06.901287666 (11h43m28.645344528s) [upload=12145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:06.901291297 (11h43m28.645348161s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.622244822 (11h43m58.366301669s) [upload=12145]: Done uploading segments... Aug 7 21:23:36.622267267 (11h43m58.366324114s) [upload=12145]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:20.942800102 (11h44m42.686856953s) [upload=12164]: Starting upload Aug 7 21:24:20.942876168 (11h44m42.686933032s) [upload=12164]: Done starting upload Aug 7 21:24:20.942953347 (11h44m42.687010195s) [upload=12164]: Uploading segments... Aug 7 21:24:20.943004925 (11h44m42.687061797s) [upload=12164]: Got next segment. Inline: true Aug 7 21:24:20.943010718 (11h44m42.687067582s) [upload=12164]: Waiting for error group managing segments... Aug 7 21:24:20.943057728 (11h44m42.687114586s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:20.943063424 (11h44m42.687120287s) [upload=12164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:20.943066967 (11h44m42.687123828s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:50.669999700 (11h45m12.414056561s) [upload=12164]: Done uploading segments... Aug 7 21:24:50.670022022 (11h45m12.414078882s) [upload=12164]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:15.759998843 (11h45m37.504055689s) [upload=12180]: Starting upload Aug 7 21:25:15.760060918 (11h45m37.504117764s) [upload=12180]: Done starting upload Aug 7 21:25:15.760121258 (11h45m37.504178104s) [upload=12180]: Uploading segments... Aug 7 21:25:15.760159734 (11h45m37.504216604s) [upload=12180]: Got next segment. Inline: true Aug 7 21:25:15.760164169 (11h45m37.504221029s) [upload=12180]: Waiting for error group managing segments... Aug 7 21:25:15.760202724 (11h45m37.504259584s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:15.760207148 (11h45m37.504264007s) [upload=12180]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:15.760212108 (11h45m37.504268971s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:45.477193191 (11h46m7.22125006s) [upload=12180]: Done uploading segments... Aug 7 21:25:45.477222897 (11h46m7.22127976s) [upload=12180]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:45.889252639 (11h46m7.633309501s) [upload=12186]: Starting upload Aug 7 21:25:45.889337905 (11h46m7.633394765s) [upload=12186]: Done starting upload Aug 7 21:25:45.889442879 (11h46m7.633499729s) [upload=12186]: Uploading segments... Aug 7 21:25:45.889501484 (11h46m7.633558347s) [upload=12186]: Got next segment. Inline: true Aug 7 21:25:45.889507484 (11h46m7.633564344s) [upload=12186]: Waiting for error group managing segments... Aug 7 21:25:45.889551118 (11h46m7.633607968s) [upload=12186]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:45.889556440 (11h46m7.633613289s) [upload=12186]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:45.889559845 (11h46m7.633616705s) [upload=12186]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:15.619717489 (11h46m37.363774339s) [upload=12186]: Done uploading segments... Aug 7 21:26:15.619739739 (11h46m37.363796589s) [upload=12186]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:40.252456263 (11h47m1.996513132s) [upload=12201]: Starting upload Aug 7 21:26:40.252598168 (11h47m1.996655038s) [upload=12201]: Done starting upload Aug 7 21:26:40.252702071 (11h47m1.996758923s) [upload=12201]: Uploading segments... Aug 7 21:26:40.252782136 (11h47m1.996839003s) [upload=12201]: Got next segment. Inline: true Aug 7 21:26:40.252790092 (11h47m1.996846954s) [upload=12201]: Waiting for error group managing segments... Aug 7 21:26:40.252842463 (11h47m1.996899315s) [upload=12201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:40.252849740 (11h47m1.996906592s) [upload=12201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:40.252854153 (11h47m1.996911015s) [upload=12201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:09.981768433 (11h47m31.725825298s) [upload=12201]: Done uploading segments... Aug 7 21:27:09.981793631 (11h47m31.725850483s) [upload=12201]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:26.837217755 (11h47m48.581274619s) [upload=12211]: Starting upload Aug 7 21:27:26.837300223 (11h47m48.581357084s) [upload=12211]: Done starting upload Aug 7 21:27:26.837385984 (11h47m48.581442831s) [upload=12211]: Uploading segments... Aug 7 21:27:26.837444443 (11h47m48.581501313s) [upload=12211]: Got next segment. Inline: true Aug 7 21:27:26.837449770 (11h47m48.581506619s) [upload=12211]: Waiting for error group managing segments... Aug 7 21:27:26.837517257 (11h47m48.581574107s) [upload=12211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:26.837523154 (11h47m48.581580003s) [upload=12211]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:26.837527097 (11h47m48.58158395s) [upload=12211]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:56.564982463 (11h48m18.309039313s) [upload=12211]: Done uploading segments... Aug 7 21:27:56.565004020 (11h48m18.309060871s) [upload=12211]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:36.367399060 (11h48m58.11145591s) [upload=12226]: Starting upload Aug 7 21:28:36.367588554 (11h48m58.111645417s) [upload=12226]: Done starting upload Aug 7 21:28:36.367664912 (11h48m58.111721765s) [upload=12226]: Uploading segments... Aug 7 21:28:36.367723895 (11h48m58.111780757s) [upload=12226]: Got next segment. Inline: true Aug 7 21:28:36.367729331 (11h48m58.111786192s) [upload=12226]: Waiting for error group managing segments... Aug 7 21:28:36.367785894 (11h48m58.111842754s) [upload=12226]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:36.367791010 (11h48m58.11184787s) [upload=12226]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:36.367794492 (11h48m58.111851352s) [upload=12226]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:06.092890267 (11h49m27.836947129s) [upload=12226]: Done uploading segments... Aug 7 21:29:06.092932205 (11h49m27.836989051s) [upload=12226]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000757285s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [.....................................X..XXXXXXXXXXXXXXXXXXXX] duration: 30.001384669s FAIL (2025-08-07 21:29:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001415083s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 279.103148ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:28:57 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 279.128166ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.274551188s FAIL (2025-08-07 21:28:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== 68895 (10h54m20.003725754s) [upload=11347]: Done uploading segments... Aug 7 20:33:58.259694049 (10h54m20.00375091s) [upload=11347]: Upload finished. err: <nil> Aug 7 20:34:42.291300370 (10h55m4.035357231s) [upload=11357]: Starting upload Aug 7 20:34:42.291392440 (10h55m4.035449315s) [upload=11357]: Done starting upload Aug 7 20:34:42.291505939 (10h55m4.035562788s) [upload=11357]: Uploading segments... Aug 7 20:34:42.291587798 (10h55m4.035644667s) [upload=11357]: Got next segment. Inline: true Aug 7 20:34:42.291610428 (10h55m4.035667289s) [upload=11357]: Waiting for error group managing segments... Aug 7 20:34:42.291660078 (10h55m4.035716927s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:34:42.291665407 (10h55m4.035722257s) [upload=11357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:34:42.291668818 (10h55m4.035725679s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:34:42.445169337 (10h55m4.189226185s) [upload=11357]: Done uploading segments... Aug 7 20:34:42.445187662 (10h55m4.189244508s) [upload=11357]: Upload finished. err: <nil> Aug 7 20:35:56.993979713 (10h56m18.738036563s) [upload=11378]: Starting upload Aug 7 20:35:56.994042734 (10h56m18.738099594s) [upload=11378]: Done starting upload Aug 7 20:35:56.994133564 (10h56m18.738190418s) [upload=11378]: Uploading segments... Aug 7 20:35:56.994188836 (10h56m18.738245705s) [upload=11378]: Got next segment. Inline: true Aug 7 20:35:56.994194253 (10h56m18.738251113s) [upload=11378]: Waiting for error group managing segments... Aug 7 20:35:56.994255059 (10h56m18.738311921s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:35:56.994261721 (10h56m18.738318586s) [upload=11378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:35:56.994266228 (10h56m18.73832309s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:35:57.196915243 (10h56m18.940972094s) [upload=11378]: Done uploading segments... Aug 7 20:35:57.196933580 (10h56m18.940990431s) [upload=11378]: Upload finished. err: <nil> Aug 7 20:36:48.383926585 (10h57m10.127983431s) [upload=11392]: Starting upload Aug 7 20:36:48.383977279 (10h57m10.128034139s) [upload=11392]: Done starting upload Aug 7 20:36:48.384080928 (10h57m10.128137781s) [upload=11392]: Uploading segments... Aug 7 20:36:48.384124913 (10h57m10.128181781s) [upload=11392]: Got next segment. Inline: true Aug 7 20:36:48.384130188 (10h57m10.128187037s) [upload=11392]: Waiting for error group managing segments... Aug 7 20:36:48.384183298 (10h57m10.128240148s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:36:48.384188890 (10h57m10.128245741s) [upload=11392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:36:48.384192353 (10h57m10.128249203s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:36:48.597716681 (10h57m10.341773532s) [upload=11392]: Done uploading segments... Aug 7 20:36:48.597735626 (10h57m10.341792476s) [upload=11392]: Upload finished. err: <nil> Aug 7 20:37:59.886756865 (10h58m21.630813711s) [upload=11411]: Starting upload Aug 7 20:37:59.886810662 (10h58m21.630867522s) [upload=11411]: Done starting upload Aug 7 20:37:59.886870714 (10h58m21.630927561s) [upload=11411]: Uploading segments... Aug 7 20:37:59.886906763 (10h58m21.630963634s) [upload=11411]: Got next segment. Inline: true Aug 7 20:37:59.886911413 (10h58m21.630968259s) [upload=11411]: Waiting for error group managing segments... Aug 7 20:37:59.886948286 (10h58m21.631005132s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:37:59.886952923 (10h58m21.631009769s) [upload=11411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:37:59.886957250 (10h58m21.631014096s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:38:00.073123067 (10h58m21.817179918s) [upload=11411]: Done uploading segments... Aug 7 20:38:00.073142248 (10h58m21.817199097s) [upload=11411]: Upload finished. err: <nil> Aug 7 20:39:06.192304563 (10h59m27.936361423s) [upload=11429]: Starting upload Aug 7 20:39:06.192379945 (10h59m27.936436804s) [upload=11429]: Done starting upload Aug 7 20:39:06.192450925 (10h59m27.936507772s) [upload=11429]: Uploading segments... Aug 7 20:39:06.192493866 (10h59m27.936550726s) [upload=11429]: Got next segment. Inline: true Aug 7 20:39:06.192499400 (10h59m27.93655626s) [upload=11429]: Waiting for error group managing segments... Aug 7 20:39:06.192549058 (10h59m27.936605918s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:06.192553658 (10h59m27.936610518s) [upload=11429]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:06.192556283 (10h59m27.936613143s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:06.410732058 (10h59m28.154788918s) [upload=11429]: Done uploading segments... Aug 7 20:39:06.410751072 (10h59m28.154807921s) [upload=11429]: Upload finished. err: <nil> Aug 7 20:39:38.189437550 (10h59m59.9334944s) [upload=11438]: Starting upload Aug 7 20:39:38.189494699 (10h59m59.933551565s) [upload=11438]: Done starting upload Aug 7 20:39:38.189570885 (10h59m59.933627735s) [upload=11438]: Uploading segments... Aug 7 20:39:38.189613828 (10h59m59.933670698s) [upload=11438]: Got next segment. Inline: true Aug 7 20:39:38.189618898 (10h59m59.933675758s) [upload=11438]: Waiting for error group managing segments... Aug 7 20:39:38.189671547 (10h59m59.933728396s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:38.189676923 (10h59m59.933733774s) [upload=11438]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:38.189680363 (10h59m59.933737214s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:38.339370287 (11h0m0.083427138s) [upload=11438]: Done uploading segments... Aug 7 20:39:38.339394965 (11h0m0.083451822s) [upload=11438]: Upload finished. err: <nil> Aug 7 20:40:13.049368895 (11h0m34.793425745s) [upload=11448]: Starting upload Aug 7 20:40:13.049412890 (11h0m34.793469743s) [upload=11448]: Done starting upload Aug 7 20:40:13.049490173 (11h0m34.793547027s) [upload=11448]: Uploading segments... Aug 7 20:40:13.049531553 (11h0m34.793588423s) [upload=11448]: Got next segment. Inline: true Aug 7 20:40:13.049536305 (11h0m34.79359316s) [upload=11448]: Waiting for error group managing segments... Aug 7 20:40:13.049606089 (11h0m34.793662937s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:40:13.049611863 (11h0m34.793668715s) [upload=11448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:40:13.049615487 (11h0m34.793672338s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:40:13.251485869 (11h0m34.995542721s) [upload=11448]: Done uploading segments... Aug 7 20:40:13.251504252 (11h0m34.995561102s) [upload=11448]: Upload finished. err: <nil> Aug 7 20:41:01.300323709 (11h1m23.044380561s) [upload=11462]: Starting upload Aug 7 20:41:01.300386685 (11h1m23.044443554s) [upload=11462]: Done starting upload Aug 7 20:41:01.300471424 (11h1m23.044528277s) [upload=11462]: Uploading segments... Aug 7 20:41:01.300528867 (11h1m23.044585736s) [upload=11462]: Got next segment. Inline: true Aug 7 20:41:01.300534426 (11h1m23.044591286s) [upload=11462]: Waiting for error group managing segments... Aug 7 20:41:01.300592418 (11h1m23.044649268s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:01.300598189 (11h1m23.04465505s) [upload=11462]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:01.300601616 (11h1m23.044658476s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:01.448083138 (11h1m23.192139989s) [upload=11462]: Done uploading segments... Aug 7 20:41:01.448102791 (11h1m23.192159644s) [upload=11462]: Upload finished. err: <nil> Aug 7 20:41:25.522931110 (11h1m47.266987978s) [upload=11468]: Starting upload Aug 7 20:41:25.523039051 (11h1m47.267095924s) [upload=11468]: Done starting upload Aug 7 20:41:25.523181759 (11h1m47.267238621s) [upload=11468]: Uploading segments... Aug 7 20:41:25.523403073 (11h1m47.267459944s) [upload=11468]: Got next segment. Inline: true Aug 7 20:41:25.523413059 (11h1m47.26746993s) [upload=11468]: Waiting for error group managing segments... Aug 7 20:41:25.523492519 (11h1m47.267549389s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:25.523498700 (11h1m47.267555566s) [upload=11468]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:25.523502761 (11h1m47.267559629s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:25.840530122 (11h1m47.584586983s) [upload=11468]: Done uploading segments... Aug 7 20:41:25.840544406 (11h1m47.584601253s) [upload=11468]: Upload finished. err: <nil> Aug 7 20:41:49.037046771 (11h2m10.781103621s) [upload=11474]: Starting upload Aug 7 20:41:49.037108376 (11h2m10.781165228s) [upload=11474]: Done starting upload Aug 7 20:41:49.037181045 (11h2m10.781237895s) [upload=11474]: Uploading segments... Aug 7 20:41:49.037238734 (11h2m10.781295606s) [upload=11474]: Got next segment. Inline: true Aug 7 20:41:49.037244163 (11h2m10.781301028s) [upload=11474]: Waiting for error group managing segments... Aug 7 20:41:49.037309765 (11h2m10.781366628s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:49.037314702 (11h2m10.781371562s) [upload=11474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:49.037317447 (11h2m10.781374307s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:49.205260584 (11h2m10.949317434s) [upload=11474]: Done uploading segments... Aug 7 20:41:49.205278299 (11h2m10.94933515s) [upload=11474]: Upload finished. err: <nil> Duration: 1.274579429s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X......................X...................] duration: 2.670343304s FAIL (2025-08-07 21:28:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== oad finished. err: <nil> Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments... Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment... Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment. Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments... Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces. Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments... Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil> Duration: 2.670375206s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 293.984503ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:28:41 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 294.012871ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration: 30.001409483s FAIL (2025-08-07 21:28:40 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001439715s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X......................X..................] duration: 3.0477769s FAIL (2025-08-07 21:28:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== oad finished. err: <nil> Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments... Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment... Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment. Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments... Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces. Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments... Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil> Duration: 3.047816244s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 1m0.000874334s FAIL (2025-08-07 21:28:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00090529s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 30.000751528s FAIL (2025-08-07 21:28:16 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000778247s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 13 out of 60 is failed (threshold=3) [..............................................X.XXXXXXXXXXXX] duration: 1m0.000506229s FAIL (2025-08-07 21:28:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ) [upload=12074]: Uploading segments... Aug 7 21:17:54.944771865 (11h38m16.688828736s) [upload=12074]: Got next segment. Inline: true Aug 7 21:17:54.944781252 (11h38m16.688838123s) [upload=12074]: Waiting for error group managing segments... Aug 7 21:17:54.944841042 (11h38m16.688897916s) [upload=12074]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:54.944848212 (11h38m16.688905079s) [upload=12074]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:54.944851963 (11h38m16.688908827s) [upload=12074]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:54.673864467 (11h39m16.417921315s) [upload=12074]: Done uploading segments... Aug 7 21:18:54.673885859 (11h39m16.417942708s) [upload=12074]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:55.119912759 (11h39m16.863969619s) [upload=12087]: Starting upload Aug 7 21:18:55.120013170 (11h39m16.864070018s) [upload=12087]: Done starting upload Aug 7 21:18:55.120094735 (11h39m16.864151596s) [upload=12087]: Uploading segments... Aug 7 21:18:55.120142836 (11h39m16.864199705s) [upload=12087]: Got next segment. Inline: true Aug 7 21:18:55.120147776 (11h39m16.864204637s) [upload=12087]: Waiting for error group managing segments... Aug 7 21:18:55.120201712 (11h39m16.864258573s) [upload=12087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:55.120207674 (11h39m16.864264534s) [upload=12087]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:55.120211321 (11h39m16.864268183s) [upload=12087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:54.844368358 (11h40m16.588425206s) [upload=12087]: Done uploading segments... Aug 7 21:19:54.844395070 (11h40m16.588451926s) [upload=12087]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:04.793025148 (11h40m26.537081997s) [upload=12103]: Starting upload Aug 7 21:20:04.793120317 (11h40m26.53717719s) [upload=12103]: Done starting upload Aug 7 21:20:04.793244612 (11h40m26.537301459s) [upload=12103]: Uploading segments... Aug 7 21:20:04.793331764 (11h40m26.537388638s) [upload=12103]: Got next segment. Inline: true Aug 7 21:20:04.793339748 (11h40m26.537396613s) [upload=12103]: Waiting for error group managing segments... Aug 7 21:20:04.793417981 (11h40m26.537474831s) [upload=12103]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:04.793424412 (11h40m26.537481262s) [upload=12103]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:04.793427866 (11h40m26.537484716s) [upload=12103]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:04.512178875 (11h41m26.256235737s) [upload=12103]: Done uploading segments... Aug 7 21:21:04.512205338 (11h41m26.256262189s) [upload=12103]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:14.239199000 (11h41m35.983255849s) [upload=12119]: Starting upload Aug 7 21:21:14.239282320 (11h41m35.983339189s) [upload=12119]: Done starting upload Aug 7 21:21:14.239356308 (11h41m35.98341316s) [upload=12119]: Uploading segments... Aug 7 21:21:14.239528709 (11h41m35.983585615s) [upload=12119]: Got next segment. Inline: true Aug 7 21:21:14.239538735 (11h41m35.983595603s) [upload=12119]: Waiting for error group managing segments... Aug 7 21:21:14.239621427 (11h41m35.983678288s) [upload=12119]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:14.239629030 (11h41m35.983685885s) [upload=12119]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:14.239632488 (11h41m35.98368934s) [upload=12119]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:13.962302553 (11h42m35.706359403s) [upload=12119]: Done uploading segments... Aug 7 21:22:13.962327972 (11h42m35.706384822s) [upload=12119]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:14.360206935 (11h42m36.104263785s) [upload=12134]: Starting upload Aug 7 21:22:14.360271125 (11h42m36.104327976s) [upload=12134]: Done starting upload Aug 7 21:22:14.360336747 (11h42m36.104393607s) [upload=12134]: Uploading segments... Aug 7 21:22:14.360380199 (11h42m36.104437068s) [upload=12134]: Got next segment. Inline: true Aug 7 21:22:14.360385659 (11h42m36.10444252s) [upload=12134]: Waiting for error group managing segments... Aug 7 21:22:14.360431265 (11h42m36.104488116s) [upload=12134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:14.360437033 (11h42m36.104493884s) [upload=12134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:14.360440534 (11h42m36.104497395s) [upload=12134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:14.088328715 (11h43m35.832385565s) [upload=12134]: Done uploading segments... Aug 7 21:23:14.088350462 (11h43m35.832407323s) [upload=12134]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:14.505772185 (11h43m36.249829051s) [upload=12148]: Starting upload Aug 7 21:23:14.505877531 (11h43m36.249934403s) [upload=12148]: Done starting upload Aug 7 21:23:14.505973975 (11h43m36.250030823s) [upload=12148]: Uploading segments... Aug 7 21:23:14.506132768 (11h43m36.250189638s) [upload=12148]: Got next segment. Inline: true Aug 7 21:23:14.506140538 (11h43m36.250197403s) [upload=12148]: Waiting for error group managing segments... Aug 7 21:23:14.506211738 (11h43m36.250268603s) [upload=12148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:14.506218667 (11h43m36.250275531s) [upload=12148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:14.506222979 (11h43m36.250279842s) [upload=12148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:14.230441064 (11h44m35.974497931s) [upload=12148]: Done uploading segments... Aug 7 21:24:14.230468679 (11h44m35.974525551s) [upload=12148]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:14.664138591 (11h44m36.408195463s) [upload=12162]: Starting upload Aug 7 21:24:14.664235374 (11h44m36.408292237s) [upload=12162]: Done starting upload Aug 7 21:24:14.664324426 (11h44m36.408381273s) [upload=12162]: Uploading segments... Aug 7 21:24:14.664378438 (11h44m36.408435308s) [upload=12162]: Got next segment. Inline: true Aug 7 21:24:14.664385109 (11h44m36.408441977s) [upload=12162]: Waiting for error group managing segments... Aug 7 21:24:14.664444486 (11h44m36.408501361s) [upload=12162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:14.664452219 (11h44m36.408509087s) [upload=12162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:14.664455983 (11h44m36.40851285s) [upload=12162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:14.390008254 (11h45m36.134065107s) [upload=12162]: Done uploading segments... Aug 7 21:25:14.390037098 (11h45m36.134093947s) [upload=12162]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:14.891337243 (11h45m36.635394094s) [upload=12178]: Starting upload Aug 7 21:25:14.891405973 (11h45m36.635462834s) [upload=12178]: Done starting upload Aug 7 21:25:14.891587260 (11h45m36.635644108s) [upload=12178]: Uploading segments... Aug 7 21:25:14.891659588 (11h45m36.635716456s) [upload=12178]: Got next segment. Inline: true Aug 7 21:25:14.891666181 (11h45m36.635723042s) [upload=12178]: Waiting for error group managing segments... Aug 7 21:25:14.891713596 (11h45m36.635770446s) [upload=12178]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:14.891719665 (11h45m36.635776525s) [upload=12178]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:14.891723701 (11h45m36.635780562s) [upload=12178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:14.618805682 (11h46m36.362862552s) [upload=12178]: Done uploading segments... Aug 7 21:26:14.618848987 (11h46m36.36290586s) [upload=12178]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:15.042634192 (11h46m36.786691042s) [upload=12195]: Starting upload Aug 7 21:26:15.042707477 (11h46m36.786764348s) [upload=12195]: Done starting upload Aug 7 21:26:15.042793174 (11h46m36.786850024s) [upload=12195]: Uploading segments... Aug 7 21:26:15.042848386 (11h46m36.78690526s) [upload=12195]: Got next segment. Inline: true Aug 7 21:26:15.042855793 (11h46m36.786912664s) [upload=12195]: Waiting for error group managing segments... Aug 7 21:26:15.042911164 (11h46m36.786968025s) [upload=12195]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:15.042918440 (11h46m36.786975312s) [upload=12195]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:15.042924061 (11h46m36.78698093s) [upload=12195]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:14.771724725 (11h47m36.515781576s) [upload=12195]: Done uploading segments... Aug 7 21:27:14.771758030 (11h47m36.515814885s) [upload=12195]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:15.202352221 (11h47m36.946409081s) [upload=12209]: Starting upload Aug 7 21:27:15.202440356 (11h47m36.946497217s) [upload=12209]: Done starting upload Aug 7 21:27:15.202525888 (11h47m36.946582739s) [upload=12209]: Uploading segments... Aug 7 21:27:15.202588330 (11h47m36.946645196s) [upload=12209]: Got next segment. Inline: true Aug 7 21:27:15.202594295 (11h47m36.946651156s) [upload=12209]: Waiting for error group managing segments... Aug 7 21:27:15.202651093 (11h47m36.946707944s) [upload=12209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:15.202656763 (11h47m36.946713624s) [upload=12209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:15.202660199 (11h47m36.946717059s) [upload=12209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:14.931851536 (11h48m36.675908396s) [upload=12209]: Done uploading segments... Aug 7 21:28:14.931874519 (11h48m36.67593138s) [upload=12209]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.000521677s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 17 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXXXXXXX] duration: 30.001584291s FAIL (2025-08-07 21:28:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001610591s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [.............................................XX.............] duration: 328.143231ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:28:08 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 328.180463ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 374.92605ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:27:59 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 374.955952ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 20 out of 60 is failed (threshold=3) [.......................................XX.XXXXXXXXXXXXXXXXXX] duration: 30.000716992s FAIL (2025-08-07 21:27:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h40m45.971102945s) [upload=12106]: Uploading segments... Aug 7 21:20:24.227086861 (11h40m45.97114373s) [upload=12106]: Got next segment. Inline: true Aug 7 21:20:24.227092422 (11h40m45.971149283s) [upload=12106]: Waiting for error group managing segments... Aug 7 21:20:24.227144188 (11h40m45.971201039s) [upload=12106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:24.227149758 (11h40m45.971206619s) [upload=12106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:24.227153486 (11h40m45.971210345s) [upload=12106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:53.942184977 (11h41m15.68624183s) [upload=12106]: Done uploading segments... Aug 7 21:20:53.942206917 (11h41m15.686263767s) [upload=12106]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:03.619018325 (11h41m25.363075197s) [upload=12116]: Starting upload Aug 7 21:21:03.619120575 (11h41m25.363177439s) [upload=12116]: Done starting upload Aug 7 21:21:03.619209054 (11h41m25.363265908s) [upload=12116]: Uploading segments... Aug 7 21:21:03.619303607 (11h41m25.363360476s) [upload=12116]: Got next segment. Inline: true Aug 7 21:21:03.619310139 (11h41m25.363366989s) [upload=12116]: Waiting for error group managing segments... Aug 7 21:21:03.619383781 (11h41m25.363440632s) [upload=12116]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:03.619390203 (11h41m25.363447055s) [upload=12116]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:03.619504810 (11h41m25.363561678s) [upload=12116]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:33.340185414 (11h41m55.084242267s) [upload=12116]: Done uploading segments... Aug 7 21:21:33.340212992 (11h41m55.08426986s) [upload=12116]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:33.783052975 (11h41m55.527109844s) [upload=12126]: Starting upload Aug 7 21:21:33.783141523 (11h41m55.527198395s) [upload=12126]: Done starting upload Aug 7 21:21:33.783231997 (11h41m55.527288844s) [upload=12126]: Uploading segments... Aug 7 21:21:33.783290191 (11h41m55.527347062s) [upload=12126]: Got next segment. Inline: true Aug 7 21:21:33.783297437 (11h41m55.527354312s) [upload=12126]: Waiting for error group managing segments... Aug 7 21:21:33.783352192 (11h41m55.527409054s) [upload=12126]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:33.783359317 (11h41m55.527416182s) [upload=12126]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:33.783363849 (11h41m55.527420713s) [upload=12126]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:03.508585017 (11h42m25.252641871s) [upload=12126]: Done uploading segments... Aug 7 21:22:03.508625142 (11h42m25.252681994s) [upload=12126]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:03.918704824 (11h42m25.662761676s) [upload=12132]: Starting upload Aug 7 21:22:03.918771848 (11h42m25.662828712s) [upload=12132]: Done starting upload Aug 7 21:22:03.918836144 (11h42m25.662892995s) [upload=12132]: Uploading segments... Aug 7 21:22:03.918882565 (11h42m25.662939434s) [upload=12132]: Got next segment. Inline: true Aug 7 21:22:03.918887612 (11h42m25.662944465s) [upload=12132]: Waiting for error group managing segments... Aug 7 21:22:03.918934357 (11h42m25.662991223s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:03.918943101 (11h42m25.662999966s) [upload=12132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:03.918952300 (11h42m25.663009183s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:33.649567967 (11h42m55.393624817s) [upload=12132]: Done uploading segments... Aug 7 21:22:33.649593937 (11h42m55.3936508s) [upload=12132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:06.900544850 (11h43m28.644601713s) [upload=12145]: Starting upload Aug 7 21:23:06.900612638 (11h43m28.644669489s) [upload=12145]: Done starting upload Aug 7 21:23:06.901077845 (11h43m28.645134714s) [upload=12145]: Uploading segments... Aug 7 21:23:06.901186098 (11h43m28.645242968s) [upload=12145]: Got next segment. Inline: true Aug 7 21:23:06.901209675 (11h43m28.645266536s) [upload=12145]: Waiting for error group managing segments... Aug 7 21:23:06.901265651 (11h43m28.645322518s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:06.901287666 (11h43m28.645344528s) [upload=12145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:06.901291297 (11h43m28.645348161s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.622244822 (11h43m58.366301669s) [upload=12145]: Done uploading segments... Aug 7 21:23:36.622267267 (11h43m58.366324114s) [upload=12145]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:20.942800102 (11h44m42.686856953s) [upload=12164]: Starting upload Aug 7 21:24:20.942876168 (11h44m42.686933032s) [upload=12164]: Done starting upload Aug 7 21:24:20.942953347 (11h44m42.687010195s) [upload=12164]: Uploading segments... Aug 7 21:24:20.943004925 (11h44m42.687061797s) [upload=12164]: Got next segment. Inline: true Aug 7 21:24:20.943010718 (11h44m42.687067582s) [upload=12164]: Waiting for error group managing segments... Aug 7 21:24:20.943057728 (11h44m42.687114586s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:20.943063424 (11h44m42.687120287s) [upload=12164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:20.943066967 (11h44m42.687123828s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:50.669999700 (11h45m12.414056561s) [upload=12164]: Done uploading segments... Aug 7 21:24:50.670022022 (11h45m12.414078882s) [upload=12164]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:15.759998843 (11h45m37.504055689s) [upload=12180]: Starting upload Aug 7 21:25:15.760060918 (11h45m37.504117764s) [upload=12180]: Done starting upload Aug 7 21:25:15.760121258 (11h45m37.504178104s) [upload=12180]: Uploading segments... Aug 7 21:25:15.760159734 (11h45m37.504216604s) [upload=12180]: Got next segment. Inline: true Aug 7 21:25:15.760164169 (11h45m37.504221029s) [upload=12180]: Waiting for error group managing segments... Aug 7 21:25:15.760202724 (11h45m37.504259584s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:15.760207148 (11h45m37.504264007s) [upload=12180]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:15.760212108 (11h45m37.504268971s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:45.477193191 (11h46m7.22125006s) [upload=12180]: Done uploading segments... Aug 7 21:25:45.477222897 (11h46m7.22127976s) [upload=12180]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:45.889252639 (11h46m7.633309501s) [upload=12186]: Starting upload Aug 7 21:25:45.889337905 (11h46m7.633394765s) [upload=12186]: Done starting upload Aug 7 21:25:45.889442879 (11h46m7.633499729s) [upload=12186]: Uploading segments... Aug 7 21:25:45.889501484 (11h46m7.633558347s) [upload=12186]: Got next segment. Inline: true Aug 7 21:25:45.889507484 (11h46m7.633564344s) [upload=12186]: Waiting for error group managing segments... Aug 7 21:25:45.889551118 (11h46m7.633607968s) [upload=12186]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:45.889556440 (11h46m7.633613289s) [upload=12186]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:45.889559845 (11h46m7.633616705s) [upload=12186]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:15.619717489 (11h46m37.363774339s) [upload=12186]: Done uploading segments... Aug 7 21:26:15.619739739 (11h46m37.363796589s) [upload=12186]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:40.252456263 (11h47m1.996513132s) [upload=12201]: Starting upload Aug 7 21:26:40.252598168 (11h47m1.996655038s) [upload=12201]: Done starting upload Aug 7 21:26:40.252702071 (11h47m1.996758923s) [upload=12201]: Uploading segments... Aug 7 21:26:40.252782136 (11h47m1.996839003s) [upload=12201]: Got next segment. Inline: true Aug 7 21:26:40.252790092 (11h47m1.996846954s) [upload=12201]: Waiting for error group managing segments... Aug 7 21:26:40.252842463 (11h47m1.996899315s) [upload=12201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:40.252849740 (11h47m1.996906592s) [upload=12201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:40.252854153 (11h47m1.996911015s) [upload=12201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:09.981768433 (11h47m31.725825298s) [upload=12201]: Done uploading segments... Aug 7 21:27:09.981793631 (11h47m31.725850483s) [upload=12201]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:26.837217755 (11h47m48.581274619s) [upload=12211]: Starting upload Aug 7 21:27:26.837300223 (11h47m48.581357084s) [upload=12211]: Done starting upload Aug 7 21:27:26.837385984 (11h47m48.581442831s) [upload=12211]: Uploading segments... Aug 7 21:27:26.837444443 (11h47m48.581501313s) [upload=12211]: Got next segment. Inline: true Aug 7 21:27:26.837449770 (11h47m48.581506619s) [upload=12211]: Waiting for error group managing segments... Aug 7 21:27:26.837517257 (11h47m48.581574107s) [upload=12211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:26.837523154 (11h47m48.581580003s) [upload=12211]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:26.837527097 (11h47m48.58158395s) [upload=12211]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:56.564982463 (11h48m18.309039313s) [upload=12211]: Done uploading segments... Aug 7 21:27:56.565004020 (11h48m18.309060871s) [upload=12211]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000740616s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [......................................X..XXXXXXXXXXXXXXXXXXX] duration: 30.001406895s FAIL (2025-08-07 21:27:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001430007s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 893.615689ms FAIL (2025-08-07 21:27:41 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== 68895 (10h54m20.003725754s) [upload=11347]: Done uploading segments... Aug 7 20:33:58.259694049 (10h54m20.00375091s) [upload=11347]: Upload finished. err: <nil> Aug 7 20:34:42.291300370 (10h55m4.035357231s) [upload=11357]: Starting upload Aug 7 20:34:42.291392440 (10h55m4.035449315s) [upload=11357]: Done starting upload Aug 7 20:34:42.291505939 (10h55m4.035562788s) [upload=11357]: Uploading segments... Aug 7 20:34:42.291587798 (10h55m4.035644667s) [upload=11357]: Got next segment. Inline: true Aug 7 20:34:42.291610428 (10h55m4.035667289s) [upload=11357]: Waiting for error group managing segments... Aug 7 20:34:42.291660078 (10h55m4.035716927s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:34:42.291665407 (10h55m4.035722257s) [upload=11357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:34:42.291668818 (10h55m4.035725679s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:34:42.445169337 (10h55m4.189226185s) [upload=11357]: Done uploading segments... Aug 7 20:34:42.445187662 (10h55m4.189244508s) [upload=11357]: Upload finished. err: <nil> Aug 7 20:35:56.993979713 (10h56m18.738036563s) [upload=11378]: Starting upload Aug 7 20:35:56.994042734 (10h56m18.738099594s) [upload=11378]: Done starting upload Aug 7 20:35:56.994133564 (10h56m18.738190418s) [upload=11378]: Uploading segments... Aug 7 20:35:56.994188836 (10h56m18.738245705s) [upload=11378]: Got next segment. Inline: true Aug 7 20:35:56.994194253 (10h56m18.738251113s) [upload=11378]: Waiting for error group managing segments... Aug 7 20:35:56.994255059 (10h56m18.738311921s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:35:56.994261721 (10h56m18.738318586s) [upload=11378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:35:56.994266228 (10h56m18.73832309s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:35:57.196915243 (10h56m18.940972094s) [upload=11378]: Done uploading segments... Aug 7 20:35:57.196933580 (10h56m18.940990431s) [upload=11378]: Upload finished. err: <nil> Aug 7 20:36:48.383926585 (10h57m10.127983431s) [upload=11392]: Starting upload Aug 7 20:36:48.383977279 (10h57m10.128034139s) [upload=11392]: Done starting upload Aug 7 20:36:48.384080928 (10h57m10.128137781s) [upload=11392]: Uploading segments... Aug 7 20:36:48.384124913 (10h57m10.128181781s) [upload=11392]: Got next segment. Inline: true Aug 7 20:36:48.384130188 (10h57m10.128187037s) [upload=11392]: Waiting for error group managing segments... Aug 7 20:36:48.384183298 (10h57m10.128240148s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:36:48.384188890 (10h57m10.128245741s) [upload=11392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:36:48.384192353 (10h57m10.128249203s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:36:48.597716681 (10h57m10.341773532s) [upload=11392]: Done uploading segments... Aug 7 20:36:48.597735626 (10h57m10.341792476s) [upload=11392]: Upload finished. err: <nil> Aug 7 20:37:59.886756865 (10h58m21.630813711s) [upload=11411]: Starting upload Aug 7 20:37:59.886810662 (10h58m21.630867522s) [upload=11411]: Done starting upload Aug 7 20:37:59.886870714 (10h58m21.630927561s) [upload=11411]: Uploading segments... Aug 7 20:37:59.886906763 (10h58m21.630963634s) [upload=11411]: Got next segment. Inline: true Aug 7 20:37:59.886911413 (10h58m21.630968259s) [upload=11411]: Waiting for error group managing segments... Aug 7 20:37:59.886948286 (10h58m21.631005132s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:37:59.886952923 (10h58m21.631009769s) [upload=11411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:37:59.886957250 (10h58m21.631014096s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:38:00.073123067 (10h58m21.817179918s) [upload=11411]: Done uploading segments... Aug 7 20:38:00.073142248 (10h58m21.817199097s) [upload=11411]: Upload finished. err: <nil> Aug 7 20:39:06.192304563 (10h59m27.936361423s) [upload=11429]: Starting upload Aug 7 20:39:06.192379945 (10h59m27.936436804s) [upload=11429]: Done starting upload Aug 7 20:39:06.192450925 (10h59m27.936507772s) [upload=11429]: Uploading segments... Aug 7 20:39:06.192493866 (10h59m27.936550726s) [upload=11429]: Got next segment. Inline: true Aug 7 20:39:06.192499400 (10h59m27.93655626s) [upload=11429]: Waiting for error group managing segments... Aug 7 20:39:06.192549058 (10h59m27.936605918s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:06.192553658 (10h59m27.936610518s) [upload=11429]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:06.192556283 (10h59m27.936613143s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:06.410732058 (10h59m28.154788918s) [upload=11429]: Done uploading segments... Aug 7 20:39:06.410751072 (10h59m28.154807921s) [upload=11429]: Upload finished. err: <nil> Aug 7 20:39:38.189437550 (10h59m59.9334944s) [upload=11438]: Starting upload Aug 7 20:39:38.189494699 (10h59m59.933551565s) [upload=11438]: Done starting upload Aug 7 20:39:38.189570885 (10h59m59.933627735s) [upload=11438]: Uploading segments... Aug 7 20:39:38.189613828 (10h59m59.933670698s) [upload=11438]: Got next segment. Inline: true Aug 7 20:39:38.189618898 (10h59m59.933675758s) [upload=11438]: Waiting for error group managing segments... Aug 7 20:39:38.189671547 (10h59m59.933728396s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:38.189676923 (10h59m59.933733774s) [upload=11438]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:38.189680363 (10h59m59.933737214s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:38.339370287 (11h0m0.083427138s) [upload=11438]: Done uploading segments... Aug 7 20:39:38.339394965 (11h0m0.083451822s) [upload=11438]: Upload finished. err: <nil> Aug 7 20:40:13.049368895 (11h0m34.793425745s) [upload=11448]: Starting upload Aug 7 20:40:13.049412890 (11h0m34.793469743s) [upload=11448]: Done starting upload Aug 7 20:40:13.049490173 (11h0m34.793547027s) [upload=11448]: Uploading segments... Aug 7 20:40:13.049531553 (11h0m34.793588423s) [upload=11448]: Got next segment. Inline: true Aug 7 20:40:13.049536305 (11h0m34.79359316s) [upload=11448]: Waiting for error group managing segments... Aug 7 20:40:13.049606089 (11h0m34.793662937s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:40:13.049611863 (11h0m34.793668715s) [upload=11448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:40:13.049615487 (11h0m34.793672338s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:40:13.251485869 (11h0m34.995542721s) [upload=11448]: Done uploading segments... Aug 7 20:40:13.251504252 (11h0m34.995561102s) [upload=11448]: Upload finished. err: <nil> Aug 7 20:41:01.300323709 (11h1m23.044380561s) [upload=11462]: Starting upload Aug 7 20:41:01.300386685 (11h1m23.044443554s) [upload=11462]: Done starting upload Aug 7 20:41:01.300471424 (11h1m23.044528277s) [upload=11462]: Uploading segments... Aug 7 20:41:01.300528867 (11h1m23.044585736s) [upload=11462]: Got next segment. Inline: true Aug 7 20:41:01.300534426 (11h1m23.044591286s) [upload=11462]: Waiting for error group managing segments... Aug 7 20:41:01.300592418 (11h1m23.044649268s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:01.300598189 (11h1m23.04465505s) [upload=11462]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:01.300601616 (11h1m23.044658476s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:01.448083138 (11h1m23.192139989s) [upload=11462]: Done uploading segments... Aug 7 20:41:01.448102791 (11h1m23.192159644s) [upload=11462]: Upload finished. err: <nil> Aug 7 20:41:25.522931110 (11h1m47.266987978s) [upload=11468]: Starting upload Aug 7 20:41:25.523039051 (11h1m47.267095924s) [upload=11468]: Done starting upload Aug 7 20:41:25.523181759 (11h1m47.267238621s) [upload=11468]: Uploading segments... Aug 7 20:41:25.523403073 (11h1m47.267459944s) [upload=11468]: Got next segment. Inline: true Aug 7 20:41:25.523413059 (11h1m47.26746993s) [upload=11468]: Waiting for error group managing segments... Aug 7 20:41:25.523492519 (11h1m47.267549389s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:25.523498700 (11h1m47.267555566s) [upload=11468]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:25.523502761 (11h1m47.267559629s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:25.840530122 (11h1m47.584586983s) [upload=11468]: Done uploading segments... Aug 7 20:41:25.840544406 (11h1m47.584601253s) [upload=11468]: Upload finished. err: <nil> Aug 7 20:41:49.037046771 (11h2m10.781103621s) [upload=11474]: Starting upload Aug 7 20:41:49.037108376 (11h2m10.781165228s) [upload=11474]: Done starting upload Aug 7 20:41:49.037181045 (11h2m10.781237895s) [upload=11474]: Uploading segments... Aug 7 20:41:49.037238734 (11h2m10.781295606s) [upload=11474]: Got next segment. Inline: true Aug 7 20:41:49.037244163 (11h2m10.781301028s) [upload=11474]: Waiting for error group managing segments... Aug 7 20:41:49.037309765 (11h2m10.781366628s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:49.037314702 (11h2m10.781371562s) [upload=11474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:49.037317447 (11h2m10.781374307s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:49.205260584 (11h2m10.949317434s) [upload=11474]: Done uploading segments... Aug 7 20:41:49.205278299 (11h2m10.94933515s) [upload=11474]: Upload finished. err: <nil> Duration: 893.646009ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [..............................................XX............] duration: 330.157134ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:27:38 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 330.218071ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 30.001587932s FAIL (2025-08-07 21:27:32 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001604675s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................X......................X.................] duration: 3.112875673s FAIL (2025-08-07 21:27:30 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== oad finished. err: <nil> Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments... Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment... Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment. Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments... Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces. Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments... Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil> Duration: 3.112909949s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration: 30.000535124s FAIL (2025-08-07 21:27:29 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000560998s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 1m0.001103284s FAIL (2025-08-07 21:27:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001135786s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)