Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.813513195s FAIL (2025-08-07 21:25:59 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== 68895 (10h54m20.003725754s) [upload=11347]: Done uploading segments... Aug 7 20:33:58.259694049 (10h54m20.00375091s) [upload=11347]: Upload finished. err: <nil> Aug 7 20:34:42.291300370 (10h55m4.035357231s) [upload=11357]: Starting upload Aug 7 20:34:42.291392440 (10h55m4.035449315s) [upload=11357]: Done starting upload Aug 7 20:34:42.291505939 (10h55m4.035562788s) [upload=11357]: Uploading segments... Aug 7 20:34:42.291587798 (10h55m4.035644667s) [upload=11357]: Got next segment. Inline: true Aug 7 20:34:42.291610428 (10h55m4.035667289s) [upload=11357]: Waiting for error group managing segments... Aug 7 20:34:42.291660078 (10h55m4.035716927s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:34:42.291665407 (10h55m4.035722257s) [upload=11357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:34:42.291668818 (10h55m4.035725679s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:34:42.445169337 (10h55m4.189226185s) [upload=11357]: Done uploading segments... Aug 7 20:34:42.445187662 (10h55m4.189244508s) [upload=11357]: Upload finished. err: <nil> Aug 7 20:35:56.993979713 (10h56m18.738036563s) [upload=11378]: Starting upload Aug 7 20:35:56.994042734 (10h56m18.738099594s) [upload=11378]: Done starting upload Aug 7 20:35:56.994133564 (10h56m18.738190418s) [upload=11378]: Uploading segments... Aug 7 20:35:56.994188836 (10h56m18.738245705s) [upload=11378]: Got next segment. Inline: true Aug 7 20:35:56.994194253 (10h56m18.738251113s) [upload=11378]: Waiting for error group managing segments... Aug 7 20:35:56.994255059 (10h56m18.738311921s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:35:56.994261721 (10h56m18.738318586s) [upload=11378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:35:56.994266228 (10h56m18.73832309s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:35:57.196915243 (10h56m18.940972094s) [upload=11378]: Done uploading segments... Aug 7 20:35:57.196933580 (10h56m18.940990431s) [upload=11378]: Upload finished. err: <nil> Aug 7 20:36:48.383926585 (10h57m10.127983431s) [upload=11392]: Starting upload Aug 7 20:36:48.383977279 (10h57m10.128034139s) [upload=11392]: Done starting upload Aug 7 20:36:48.384080928 (10h57m10.128137781s) [upload=11392]: Uploading segments... Aug 7 20:36:48.384124913 (10h57m10.128181781s) [upload=11392]: Got next segment. Inline: true Aug 7 20:36:48.384130188 (10h57m10.128187037s) [upload=11392]: Waiting for error group managing segments... Aug 7 20:36:48.384183298 (10h57m10.128240148s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:36:48.384188890 (10h57m10.128245741s) [upload=11392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:36:48.384192353 (10h57m10.128249203s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:36:48.597716681 (10h57m10.341773532s) [upload=11392]: Done uploading segments... Aug 7 20:36:48.597735626 (10h57m10.341792476s) [upload=11392]: Upload finished. err: <nil> Aug 7 20:37:59.886756865 (10h58m21.630813711s) [upload=11411]: Starting upload Aug 7 20:37:59.886810662 (10h58m21.630867522s) [upload=11411]: Done starting upload Aug 7 20:37:59.886870714 (10h58m21.630927561s) [upload=11411]: Uploading segments... Aug 7 20:37:59.886906763 (10h58m21.630963634s) [upload=11411]: Got next segment. Inline: true Aug 7 20:37:59.886911413 (10h58m21.630968259s) [upload=11411]: Waiting for error group managing segments... Aug 7 20:37:59.886948286 (10h58m21.631005132s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:37:59.886952923 (10h58m21.631009769s) [upload=11411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:37:59.886957250 (10h58m21.631014096s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:38:00.073123067 (10h58m21.817179918s) [upload=11411]: Done uploading segments... Aug 7 20:38:00.073142248 (10h58m21.817199097s) [upload=11411]: Upload finished. err: <nil> Aug 7 20:39:06.192304563 (10h59m27.936361423s) [upload=11429]: Starting upload Aug 7 20:39:06.192379945 (10h59m27.936436804s) [upload=11429]: Done starting upload Aug 7 20:39:06.192450925 (10h59m27.936507772s) [upload=11429]: Uploading segments... Aug 7 20:39:06.192493866 (10h59m27.936550726s) [upload=11429]: Got next segment. Inline: true Aug 7 20:39:06.192499400 (10h59m27.93655626s) [upload=11429]: Waiting for error group managing segments... Aug 7 20:39:06.192549058 (10h59m27.936605918s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:06.192553658 (10h59m27.936610518s) [upload=11429]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:06.192556283 (10h59m27.936613143s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:06.410732058 (10h59m28.154788918s) [upload=11429]: Done uploading segments... Aug 7 20:39:06.410751072 (10h59m28.154807921s) [upload=11429]: Upload finished. err: <nil> Aug 7 20:39:38.189437550 (10h59m59.9334944s) [upload=11438]: Starting upload Aug 7 20:39:38.189494699 (10h59m59.933551565s) [upload=11438]: Done starting upload Aug 7 20:39:38.189570885 (10h59m59.933627735s) [upload=11438]: Uploading segments... Aug 7 20:39:38.189613828 (10h59m59.933670698s) [upload=11438]: Got next segment. Inline: true Aug 7 20:39:38.189618898 (10h59m59.933675758s) [upload=11438]: Waiting for error group managing segments... Aug 7 20:39:38.189671547 (10h59m59.933728396s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:38.189676923 (10h59m59.933733774s) [upload=11438]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:38.189680363 (10h59m59.933737214s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:38.339370287 (11h0m0.083427138s) [upload=11438]: Done uploading segments... Aug 7 20:39:38.339394965 (11h0m0.083451822s) [upload=11438]: Upload finished. err: <nil> Aug 7 20:40:13.049368895 (11h0m34.793425745s) [upload=11448]: Starting upload Aug 7 20:40:13.049412890 (11h0m34.793469743s) [upload=11448]: Done starting upload Aug 7 20:40:13.049490173 (11h0m34.793547027s) [upload=11448]: Uploading segments... Aug 7 20:40:13.049531553 (11h0m34.793588423s) [upload=11448]: Got next segment. Inline: true Aug 7 20:40:13.049536305 (11h0m34.79359316s) [upload=11448]: Waiting for error group managing segments... Aug 7 20:40:13.049606089 (11h0m34.793662937s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:40:13.049611863 (11h0m34.793668715s) [upload=11448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:40:13.049615487 (11h0m34.793672338s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:40:13.251485869 (11h0m34.995542721s) [upload=11448]: Done uploading segments... Aug 7 20:40:13.251504252 (11h0m34.995561102s) [upload=11448]: Upload finished. err: <nil> Aug 7 20:41:01.300323709 (11h1m23.044380561s) [upload=11462]: Starting upload Aug 7 20:41:01.300386685 (11h1m23.044443554s) [upload=11462]: Done starting upload Aug 7 20:41:01.300471424 (11h1m23.044528277s) [upload=11462]: Uploading segments... Aug 7 20:41:01.300528867 (11h1m23.044585736s) [upload=11462]: Got next segment. Inline: true Aug 7 20:41:01.300534426 (11h1m23.044591286s) [upload=11462]: Waiting for error group managing segments... Aug 7 20:41:01.300592418 (11h1m23.044649268s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:01.300598189 (11h1m23.04465505s) [upload=11462]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:01.300601616 (11h1m23.044658476s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:01.448083138 (11h1m23.192139989s) [upload=11462]: Done uploading segments... Aug 7 20:41:01.448102791 (11h1m23.192159644s) [upload=11462]: Upload finished. err: <nil> Aug 7 20:41:25.522931110 (11h1m47.266987978s) [upload=11468]: Starting upload Aug 7 20:41:25.523039051 (11h1m47.267095924s) [upload=11468]: Done starting upload Aug 7 20:41:25.523181759 (11h1m47.267238621s) [upload=11468]: Uploading segments... Aug 7 20:41:25.523403073 (11h1m47.267459944s) [upload=11468]: Got next segment. Inline: true Aug 7 20:41:25.523413059 (11h1m47.26746993s) [upload=11468]: Waiting for error group managing segments... Aug 7 20:41:25.523492519 (11h1m47.267549389s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:25.523498700 (11h1m47.267555566s) [upload=11468]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:25.523502761 (11h1m47.267559629s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:25.840530122 (11h1m47.584586983s) [upload=11468]: Done uploading segments... Aug 7 20:41:25.840544406 (11h1m47.584601253s) [upload=11468]: Upload finished. err: <nil> Aug 7 20:41:49.037046771 (11h2m10.781103621s) [upload=11474]: Starting upload Aug 7 20:41:49.037108376 (11h2m10.781165228s) [upload=11474]: Done starting upload Aug 7 20:41:49.037181045 (11h2m10.781237895s) [upload=11474]: Uploading segments... Aug 7 20:41:49.037238734 (11h2m10.781295606s) [upload=11474]: Got next segment. Inline: true Aug 7 20:41:49.037244163 (11h2m10.781301028s) [upload=11474]: Waiting for error group managing segments... Aug 7 20:41:49.037309765 (11h2m10.781366628s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:49.037314702 (11h2m10.781371562s) [upload=11474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:49.037317447 (11h2m10.781374307s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:49.205260584 (11h2m10.949317434s) [upload=11474]: Done uploading segments... Aug 7 20:41:49.205278299 (11h2m10.94933515s) [upload=11474]: Upload finished. err: <nil> Duration: 2.813564865s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [................................................XX..........] duration: 612.769273ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:25:56 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 612.794296ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................X......................X...............] duration: 2.730975439s FAIL (2025-08-07 21:25:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== oad finished. err: <nil> Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments... Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment... Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment. Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments... Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces. Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments... Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil> Duration: 2.731007966s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 17 out of 60 is failed (threshold=3) [..........................................XX.XXXXXXXXXXXXXXX] duration: 30.00074299s FAIL (2025-08-07 21:25:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 154s) [upload=12078]: Uploading segments... Aug 7 21:18:11.676235479 (11h38m33.420292347s) [upload=12078]: Got next segment. Inline: true Aug 7 21:18:11.676241644 (11h38m33.420298508s) [upload=12078]: Waiting for error group managing segments... Aug 7 21:18:11.676304521 (11h38m33.420361386s) [upload=12078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:11.676310489 (11h38m33.420367352s) [upload=12078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:11.676314037 (11h38m33.420370906s) [upload=12078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:41.388891420 (11h39m3.132948271s) [upload=12078]: Done uploading segments... Aug 7 21:18:41.388921897 (11h39m3.132978749s) [upload=12078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:53.498741452 (11h39m15.242798302s) [upload=12086]: Starting upload Aug 7 21:18:53.498798950 (11h39m15.242855819s) [upload=12086]: Done starting upload Aug 7 21:18:53.498860950 (11h39m15.242917802s) [upload=12086]: Uploading segments... Aug 7 21:18:53.498901325 (11h39m15.242958195s) [upload=12086]: Got next segment. Inline: true Aug 7 21:18:53.498908406 (11h39m15.242965267s) [upload=12086]: Waiting for error group managing segments... Aug 7 21:18:53.498951359 (11h39m15.24300821s) [upload=12086]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:53.498956808 (11h39m15.243013668s) [upload=12086]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:53.498960205 (11h39m15.243017065s) [upload=12086]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:23.226932815 (11h39m44.970989679s) [upload=12086]: Done uploading segments... Aug 7 21:19:23.226954988 (11h39m44.971011848s) [upload=12086]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:19:23.633524671 (11h39m45.37758152s) [upload=12093]: Starting upload Aug 7 21:19:23.633599963 (11h39m45.377656832s) [upload=12093]: Done starting upload Aug 7 21:19:23.633677463 (11h39m45.377734313s) [upload=12093]: Uploading segments... Aug 7 21:19:23.633732206 (11h39m45.377789075s) [upload=12093]: Got next segment. Inline: true Aug 7 21:19:23.633737928 (11h39m45.377794778s) [upload=12093]: Waiting for error group managing segments... Aug 7 21:19:23.633790184 (11h39m45.377847035s) [upload=12093]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:19:23.633795686 (11h39m45.377852546s) [upload=12093]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:19:23.633801264 (11h39m45.377858124s) [upload=12093]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:53.363781250 (11h40m15.107838111s) [upload=12093]: Done uploading segments... Aug 7 21:19:53.363801573 (11h40m15.107858434s) [upload=12093]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:24.226894126 (11h40m45.970950987s) [upload=12106]: Starting upload Aug 7 21:20:24.226954589 (11h40m45.971011441s) [upload=12106]: Done starting upload Aug 7 21:20:24.227046095 (11h40m45.971102945s) [upload=12106]: Uploading segments... Aug 7 21:20:24.227086861 (11h40m45.97114373s) [upload=12106]: Got next segment. Inline: true Aug 7 21:20:24.227092422 (11h40m45.971149283s) [upload=12106]: Waiting for error group managing segments... Aug 7 21:20:24.227144188 (11h40m45.971201039s) [upload=12106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:24.227149758 (11h40m45.971206619s) [upload=12106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:24.227153486 (11h40m45.971210345s) [upload=12106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:53.942184977 (11h41m15.68624183s) [upload=12106]: Done uploading segments... Aug 7 21:20:53.942206917 (11h41m15.686263767s) [upload=12106]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:03.619018325 (11h41m25.363075197s) [upload=12116]: Starting upload Aug 7 21:21:03.619120575 (11h41m25.363177439s) [upload=12116]: Done starting upload Aug 7 21:21:03.619209054 (11h41m25.363265908s) [upload=12116]: Uploading segments... Aug 7 21:21:03.619303607 (11h41m25.363360476s) [upload=12116]: Got next segment. Inline: true Aug 7 21:21:03.619310139 (11h41m25.363366989s) [upload=12116]: Waiting for error group managing segments... Aug 7 21:21:03.619383781 (11h41m25.363440632s) [upload=12116]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:03.619390203 (11h41m25.363447055s) [upload=12116]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:03.619504810 (11h41m25.363561678s) [upload=12116]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:33.340185414 (11h41m55.084242267s) [upload=12116]: Done uploading segments... Aug 7 21:21:33.340212992 (11h41m55.08426986s) [upload=12116]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:33.783052975 (11h41m55.527109844s) [upload=12126]: Starting upload Aug 7 21:21:33.783141523 (11h41m55.527198395s) [upload=12126]: Done starting upload Aug 7 21:21:33.783231997 (11h41m55.527288844s) [upload=12126]: Uploading segments... Aug 7 21:21:33.783290191 (11h41m55.527347062s) [upload=12126]: Got next segment. Inline: true Aug 7 21:21:33.783297437 (11h41m55.527354312s) [upload=12126]: Waiting for error group managing segments... Aug 7 21:21:33.783352192 (11h41m55.527409054s) [upload=12126]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:33.783359317 (11h41m55.527416182s) [upload=12126]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:33.783363849 (11h41m55.527420713s) [upload=12126]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:03.508585017 (11h42m25.252641871s) [upload=12126]: Done uploading segments... Aug 7 21:22:03.508625142 (11h42m25.252681994s) [upload=12126]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:03.918704824 (11h42m25.662761676s) [upload=12132]: Starting upload Aug 7 21:22:03.918771848 (11h42m25.662828712s) [upload=12132]: Done starting upload Aug 7 21:22:03.918836144 (11h42m25.662892995s) [upload=12132]: Uploading segments... Aug 7 21:22:03.918882565 (11h42m25.662939434s) [upload=12132]: Got next segment. Inline: true Aug 7 21:22:03.918887612 (11h42m25.662944465s) [upload=12132]: Waiting for error group managing segments... Aug 7 21:22:03.918934357 (11h42m25.662991223s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:03.918943101 (11h42m25.662999966s) [upload=12132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:03.918952300 (11h42m25.663009183s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:33.649567967 (11h42m55.393624817s) [upload=12132]: Done uploading segments... Aug 7 21:22:33.649593937 (11h42m55.3936508s) [upload=12132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:06.900544850 (11h43m28.644601713s) [upload=12145]: Starting upload Aug 7 21:23:06.900612638 (11h43m28.644669489s) [upload=12145]: Done starting upload Aug 7 21:23:06.901077845 (11h43m28.645134714s) [upload=12145]: Uploading segments... Aug 7 21:23:06.901186098 (11h43m28.645242968s) [upload=12145]: Got next segment. Inline: true Aug 7 21:23:06.901209675 (11h43m28.645266536s) [upload=12145]: Waiting for error group managing segments... Aug 7 21:23:06.901265651 (11h43m28.645322518s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:06.901287666 (11h43m28.645344528s) [upload=12145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:06.901291297 (11h43m28.645348161s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.622244822 (11h43m58.366301669s) [upload=12145]: Done uploading segments... Aug 7 21:23:36.622267267 (11h43m58.366324114s) [upload=12145]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:20.942800102 (11h44m42.686856953s) [upload=12164]: Starting upload Aug 7 21:24:20.942876168 (11h44m42.686933032s) [upload=12164]: Done starting upload Aug 7 21:24:20.942953347 (11h44m42.687010195s) [upload=12164]: Uploading segments... Aug 7 21:24:20.943004925 (11h44m42.687061797s) [upload=12164]: Got next segment. Inline: true Aug 7 21:24:20.943010718 (11h44m42.687067582s) [upload=12164]: Waiting for error group managing segments... Aug 7 21:24:20.943057728 (11h44m42.687114586s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:20.943063424 (11h44m42.687120287s) [upload=12164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:20.943066967 (11h44m42.687123828s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:50.669999700 (11h45m12.414056561s) [upload=12164]: Done uploading segments... Aug 7 21:24:50.670022022 (11h45m12.414078882s) [upload=12164]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:15.759998843 (11h45m37.504055689s) [upload=12180]: Starting upload Aug 7 21:25:15.760060918 (11h45m37.504117764s) [upload=12180]: Done starting upload Aug 7 21:25:15.760121258 (11h45m37.504178104s) [upload=12180]: Uploading segments... Aug 7 21:25:15.760159734 (11h45m37.504216604s) [upload=12180]: Got next segment. Inline: true Aug 7 21:25:15.760164169 (11h45m37.504221029s) [upload=12180]: Waiting for error group managing segments... Aug 7 21:25:15.760202724 (11h45m37.504259584s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:15.760207148 (11h45m37.504264007s) [upload=12180]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:15.760212108 (11h45m37.504268971s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:45.477193191 (11h46m7.22125006s) [upload=12180]: Done uploading segments... Aug 7 21:25:45.477222897 (11h46m7.22127976s) [upload=12180]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000767916s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 30.00056508s FAIL (2025-08-07 21:25:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000659056s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 30.001307182s FAIL (2025-08-07 21:25:37 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001328664s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [........................................X..XXXXXXXXXXXXXXXXX] duration: 30.001103159s FAIL (2025-08-07 21:25:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001142228s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 15 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXXXXXX] duration: 30.000695481s FAIL (2025-08-07 21:25:29 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000724942s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................X......................X..............] duration: 2.457338947s FAIL (2025-08-07 21:25:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== oad finished. err: <nil> Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments... Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment... Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment. Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments... Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces. Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments... Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil> Duration: 2.457397742s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 10 out of 60 is failed (threshold=3) [.................................................X.XXXXXXXXX] duration: 1m0.000625015s FAIL (2025-08-07 21:25:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 01.394373467 (11h35m23.138430318s) [upload=12030]: Uploading segments... Aug 7 21:15:01.394459055 (11h35m23.138515928s) [upload=12030]: Got next segment. Inline: true Aug 7 21:15:01.394467128 (11h35m23.138523996s) [upload=12030]: Waiting for error group managing segments... Aug 7 21:15:01.394548041 (11h35m23.138604915s) [upload=12030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:01.394554816 (11h35m23.13861169s) [upload=12030]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:01.394559518 (11h35m23.138616392s) [upload=12030]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:03.584668408 (11h35m25.328725286s) [upload=12030]: Done uploading segments... Aug 7 21:15:03.584713689 (11h35m25.32877056s) [upload=12030]: Upload finished. err: <nil> Aug 7 21:15:54.636790163 (11h36m16.380847013s) [upload=12044]: Starting upload Aug 7 21:15:54.636877707 (11h36m16.380934638s) [upload=12044]: Done starting upload Aug 7 21:15:54.636936350 (11h36m16.3809932s) [upload=12044]: Uploading segments... Aug 7 21:15:54.636993824 (11h36m16.381050685s) [upload=12044]: Got next segment. Inline: true Aug 7 21:15:54.636999384 (11h36m16.381056244s) [upload=12044]: Waiting for error group managing segments... Aug 7 21:15:54.637044877 (11h36m16.381101727s) [upload=12044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:54.637050662 (11h36m16.381107522s) [upload=12044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:54.637053905 (11h36m16.381110766s) [upload=12044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:54.359175610 (11h37m16.103232476s) [upload=12044]: Done uploading segments... Aug 7 21:16:54.359203156 (11h37m16.103260018s) [upload=12044]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:54.796260076 (11h37m16.54031694s) [upload=12060]: Starting upload Aug 7 21:16:54.796405798 (11h37m16.540462658s) [upload=12060]: Done starting upload Aug 7 21:16:54.796514557 (11h37m16.540571407s) [upload=12060]: Uploading segments... Aug 7 21:16:54.796581012 (11h37m16.540637875s) [upload=12060]: Got next segment. Inline: true Aug 7 21:16:54.796586298 (11h37m16.540643158s) [upload=12060]: Waiting for error group managing segments... Aug 7 21:16:54.796657533 (11h37m16.540714383s) [upload=12060]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:54.796663030 (11h37m16.540719894s) [upload=12060]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:54.796669424 (11h37m16.540726289s) [upload=12060]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:17:54.519327140 (11h38m16.263384004s) [upload=12060]: Done uploading segments... Aug 7 21:17:54.519368022 (11h38m16.263424882s) [upload=12060]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:54.944498267 (11h38m16.688555136s) [upload=12074]: Starting upload Aug 7 21:17:54.944617406 (11h38m16.688674267s) [upload=12074]: Done starting upload Aug 7 21:17:54.944695430 (11h38m16.688752285s) [upload=12074]: Uploading segments... Aug 7 21:17:54.944771865 (11h38m16.688828736s) [upload=12074]: Got next segment. Inline: true Aug 7 21:17:54.944781252 (11h38m16.688838123s) [upload=12074]: Waiting for error group managing segments... Aug 7 21:17:54.944841042 (11h38m16.688897916s) [upload=12074]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:54.944848212 (11h38m16.688905079s) [upload=12074]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:54.944851963 (11h38m16.688908827s) [upload=12074]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:54.673864467 (11h39m16.417921315s) [upload=12074]: Done uploading segments... Aug 7 21:18:54.673885859 (11h39m16.417942708s) [upload=12074]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:55.119912759 (11h39m16.863969619s) [upload=12087]: Starting upload Aug 7 21:18:55.120013170 (11h39m16.864070018s) [upload=12087]: Done starting upload Aug 7 21:18:55.120094735 (11h39m16.864151596s) [upload=12087]: Uploading segments... Aug 7 21:18:55.120142836 (11h39m16.864199705s) [upload=12087]: Got next segment. Inline: true Aug 7 21:18:55.120147776 (11h39m16.864204637s) [upload=12087]: Waiting for error group managing segments... Aug 7 21:18:55.120201712 (11h39m16.864258573s) [upload=12087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:55.120207674 (11h39m16.864264534s) [upload=12087]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:55.120211321 (11h39m16.864268183s) [upload=12087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:54.844368358 (11h40m16.588425206s) [upload=12087]: Done uploading segments... Aug 7 21:19:54.844395070 (11h40m16.588451926s) [upload=12087]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:04.793025148 (11h40m26.537081997s) [upload=12103]: Starting upload Aug 7 21:20:04.793120317 (11h40m26.53717719s) [upload=12103]: Done starting upload Aug 7 21:20:04.793244612 (11h40m26.537301459s) [upload=12103]: Uploading segments... Aug 7 21:20:04.793331764 (11h40m26.537388638s) [upload=12103]: Got next segment. Inline: true Aug 7 21:20:04.793339748 (11h40m26.537396613s) [upload=12103]: Waiting for error group managing segments... Aug 7 21:20:04.793417981 (11h40m26.537474831s) [upload=12103]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:04.793424412 (11h40m26.537481262s) [upload=12103]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:04.793427866 (11h40m26.537484716s) [upload=12103]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:04.512178875 (11h41m26.256235737s) [upload=12103]: Done uploading segments... Aug 7 21:21:04.512205338 (11h41m26.256262189s) [upload=12103]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:14.239199000 (11h41m35.983255849s) [upload=12119]: Starting upload Aug 7 21:21:14.239282320 (11h41m35.983339189s) [upload=12119]: Done starting upload Aug 7 21:21:14.239356308 (11h41m35.98341316s) [upload=12119]: Uploading segments... Aug 7 21:21:14.239528709 (11h41m35.983585615s) [upload=12119]: Got next segment. Inline: true Aug 7 21:21:14.239538735 (11h41m35.983595603s) [upload=12119]: Waiting for error group managing segments... Aug 7 21:21:14.239621427 (11h41m35.983678288s) [upload=12119]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:14.239629030 (11h41m35.983685885s) [upload=12119]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:14.239632488 (11h41m35.98368934s) [upload=12119]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:13.962302553 (11h42m35.706359403s) [upload=12119]: Done uploading segments... Aug 7 21:22:13.962327972 (11h42m35.706384822s) [upload=12119]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:14.360206935 (11h42m36.104263785s) [upload=12134]: Starting upload Aug 7 21:22:14.360271125 (11h42m36.104327976s) [upload=12134]: Done starting upload Aug 7 21:22:14.360336747 (11h42m36.104393607s) [upload=12134]: Uploading segments... Aug 7 21:22:14.360380199 (11h42m36.104437068s) [upload=12134]: Got next segment. Inline: true Aug 7 21:22:14.360385659 (11h42m36.10444252s) [upload=12134]: Waiting for error group managing segments... Aug 7 21:22:14.360431265 (11h42m36.104488116s) [upload=12134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:14.360437033 (11h42m36.104493884s) [upload=12134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:14.360440534 (11h42m36.104497395s) [upload=12134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:14.088328715 (11h43m35.832385565s) [upload=12134]: Done uploading segments... Aug 7 21:23:14.088350462 (11h43m35.832407323s) [upload=12134]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:14.505772185 (11h43m36.249829051s) [upload=12148]: Starting upload Aug 7 21:23:14.505877531 (11h43m36.249934403s) [upload=12148]: Done starting upload Aug 7 21:23:14.505973975 (11h43m36.250030823s) [upload=12148]: Uploading segments... Aug 7 21:23:14.506132768 (11h43m36.250189638s) [upload=12148]: Got next segment. Inline: true Aug 7 21:23:14.506140538 (11h43m36.250197403s) [upload=12148]: Waiting for error group managing segments... Aug 7 21:23:14.506211738 (11h43m36.250268603s) [upload=12148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:14.506218667 (11h43m36.250275531s) [upload=12148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:14.506222979 (11h43m36.250279842s) [upload=12148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:14.230441064 (11h44m35.974497931s) [upload=12148]: Done uploading segments... Aug 7 21:24:14.230468679 (11h44m35.974525551s) [upload=12148]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:14.664138591 (11h44m36.408195463s) [upload=12162]: Starting upload Aug 7 21:24:14.664235374 (11h44m36.408292237s) [upload=12162]: Done starting upload Aug 7 21:24:14.664324426 (11h44m36.408381273s) [upload=12162]: Uploading segments... Aug 7 21:24:14.664378438 (11h44m36.408435308s) [upload=12162]: Got next segment. Inline: true Aug 7 21:24:14.664385109 (11h44m36.408441977s) [upload=12162]: Waiting for error group managing segments... Aug 7 21:24:14.664444486 (11h44m36.408501361s) [upload=12162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:14.664452219 (11h44m36.408509087s) [upload=12162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:14.664455983 (11h44m36.40851285s) [upload=12162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:14.390008254 (11h45m36.134065107s) [upload=12162]: Done uploading segments... Aug 7 21:25:14.390037098 (11h45m36.134093947s) [upload=12162]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.000659658s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 1m0.001113151s FAIL (2025-08-07 21:25:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001130529s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 290.328732ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:25:07 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 290.358267ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................X......................X.............] duration: 2.53221956s FAIL (2025-08-07 21:25:04 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== oad finished. err: <nil> Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments... Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment... Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment. Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments... Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces. Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments... Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil> Duration: 2.532309091s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 30.000699436s FAIL (2025-08-07 21:24:57 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000726728s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 16 out of 60 is failed (threshold=3) [...........................................XX.XXXXXXXXXXXXXX] duration: 30.001434295s FAIL (2025-08-07 21:24:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 3175318s) [upload=12072]: Uploading segments... Aug 7 21:17:41.499176424 (11h38m3.243233293s) [upload=12072]: Got next segment. Inline: true Aug 7 21:17:41.499181917 (11h38m3.243238777s) [upload=12072]: Waiting for error group managing segments... Aug 7 21:17:41.499236606 (11h38m3.243293472s) [upload=12072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:41.499242465 (11h38m3.24329933s) [upload=12072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:41.499245854 (11h38m3.243302721s) [upload=12072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:11.231025141 (11h38m32.97508201s) [upload=12072]: Done uploading segments... Aug 7 21:18:11.231066796 (11h38m32.975123655s) [upload=12072]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:11.675991066 (11h38m33.420047932s) [upload=12078]: Starting upload Aug 7 21:18:11.676081621 (11h38m33.42013848s) [upload=12078]: Done starting upload Aug 7 21:18:11.676172293 (11h38m33.420229154s) [upload=12078]: Uploading segments... Aug 7 21:18:11.676235479 (11h38m33.420292347s) [upload=12078]: Got next segment. Inline: true Aug 7 21:18:11.676241644 (11h38m33.420298508s) [upload=12078]: Waiting for error group managing segments... Aug 7 21:18:11.676304521 (11h38m33.420361386s) [upload=12078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:11.676310489 (11h38m33.420367352s) [upload=12078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:11.676314037 (11h38m33.420370906s) [upload=12078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:41.388891420 (11h39m3.132948271s) [upload=12078]: Done uploading segments... Aug 7 21:18:41.388921897 (11h39m3.132978749s) [upload=12078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:53.498741452 (11h39m15.242798302s) [upload=12086]: Starting upload Aug 7 21:18:53.498798950 (11h39m15.242855819s) [upload=12086]: Done starting upload Aug 7 21:18:53.498860950 (11h39m15.242917802s) [upload=12086]: Uploading segments... Aug 7 21:18:53.498901325 (11h39m15.242958195s) [upload=12086]: Got next segment. Inline: true Aug 7 21:18:53.498908406 (11h39m15.242965267s) [upload=12086]: Waiting for error group managing segments... Aug 7 21:18:53.498951359 (11h39m15.24300821s) [upload=12086]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:53.498956808 (11h39m15.243013668s) [upload=12086]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:53.498960205 (11h39m15.243017065s) [upload=12086]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:23.226932815 (11h39m44.970989679s) [upload=12086]: Done uploading segments... Aug 7 21:19:23.226954988 (11h39m44.971011848s) [upload=12086]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:19:23.633524671 (11h39m45.37758152s) [upload=12093]: Starting upload Aug 7 21:19:23.633599963 (11h39m45.377656832s) [upload=12093]: Done starting upload Aug 7 21:19:23.633677463 (11h39m45.377734313s) [upload=12093]: Uploading segments... Aug 7 21:19:23.633732206 (11h39m45.377789075s) [upload=12093]: Got next segment. Inline: true Aug 7 21:19:23.633737928 (11h39m45.377794778s) [upload=12093]: Waiting for error group managing segments... Aug 7 21:19:23.633790184 (11h39m45.377847035s) [upload=12093]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:19:23.633795686 (11h39m45.377852546s) [upload=12093]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:19:23.633801264 (11h39m45.377858124s) [upload=12093]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:53.363781250 (11h40m15.107838111s) [upload=12093]: Done uploading segments... Aug 7 21:19:53.363801573 (11h40m15.107858434s) [upload=12093]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:24.226894126 (11h40m45.970950987s) [upload=12106]: Starting upload Aug 7 21:20:24.226954589 (11h40m45.971011441s) [upload=12106]: Done starting upload Aug 7 21:20:24.227046095 (11h40m45.971102945s) [upload=12106]: Uploading segments... Aug 7 21:20:24.227086861 (11h40m45.97114373s) [upload=12106]: Got next segment. Inline: true Aug 7 21:20:24.227092422 (11h40m45.971149283s) [upload=12106]: Waiting for error group managing segments... Aug 7 21:20:24.227144188 (11h40m45.971201039s) [upload=12106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:24.227149758 (11h40m45.971206619s) [upload=12106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:24.227153486 (11h40m45.971210345s) [upload=12106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:53.942184977 (11h41m15.68624183s) [upload=12106]: Done uploading segments... Aug 7 21:20:53.942206917 (11h41m15.686263767s) [upload=12106]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:03.619018325 (11h41m25.363075197s) [upload=12116]: Starting upload Aug 7 21:21:03.619120575 (11h41m25.363177439s) [upload=12116]: Done starting upload Aug 7 21:21:03.619209054 (11h41m25.363265908s) [upload=12116]: Uploading segments... Aug 7 21:21:03.619303607 (11h41m25.363360476s) [upload=12116]: Got next segment. Inline: true Aug 7 21:21:03.619310139 (11h41m25.363366989s) [upload=12116]: Waiting for error group managing segments... Aug 7 21:21:03.619383781 (11h41m25.363440632s) [upload=12116]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:03.619390203 (11h41m25.363447055s) [upload=12116]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:03.619504810 (11h41m25.363561678s) [upload=12116]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:33.340185414 (11h41m55.084242267s) [upload=12116]: Done uploading segments... Aug 7 21:21:33.340212992 (11h41m55.08426986s) [upload=12116]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:33.783052975 (11h41m55.527109844s) [upload=12126]: Starting upload Aug 7 21:21:33.783141523 (11h41m55.527198395s) [upload=12126]: Done starting upload Aug 7 21:21:33.783231997 (11h41m55.527288844s) [upload=12126]: Uploading segments... Aug 7 21:21:33.783290191 (11h41m55.527347062s) [upload=12126]: Got next segment. Inline: true Aug 7 21:21:33.783297437 (11h41m55.527354312s) [upload=12126]: Waiting for error group managing segments... Aug 7 21:21:33.783352192 (11h41m55.527409054s) [upload=12126]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:33.783359317 (11h41m55.527416182s) [upload=12126]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:33.783363849 (11h41m55.527420713s) [upload=12126]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:03.508585017 (11h42m25.252641871s) [upload=12126]: Done uploading segments... Aug 7 21:22:03.508625142 (11h42m25.252681994s) [upload=12126]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:03.918704824 (11h42m25.662761676s) [upload=12132]: Starting upload Aug 7 21:22:03.918771848 (11h42m25.662828712s) [upload=12132]: Done starting upload Aug 7 21:22:03.918836144 (11h42m25.662892995s) [upload=12132]: Uploading segments... Aug 7 21:22:03.918882565 (11h42m25.662939434s) [upload=12132]: Got next segment. Inline: true Aug 7 21:22:03.918887612 (11h42m25.662944465s) [upload=12132]: Waiting for error group managing segments... Aug 7 21:22:03.918934357 (11h42m25.662991223s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:03.918943101 (11h42m25.662999966s) [upload=12132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:03.918952300 (11h42m25.663009183s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:33.649567967 (11h42m55.393624817s) [upload=12132]: Done uploading segments... Aug 7 21:22:33.649593937 (11h42m55.3936508s) [upload=12132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:06.900544850 (11h43m28.644601713s) [upload=12145]: Starting upload Aug 7 21:23:06.900612638 (11h43m28.644669489s) [upload=12145]: Done starting upload Aug 7 21:23:06.901077845 (11h43m28.645134714s) [upload=12145]: Uploading segments... Aug 7 21:23:06.901186098 (11h43m28.645242968s) [upload=12145]: Got next segment. Inline: true Aug 7 21:23:06.901209675 (11h43m28.645266536s) [upload=12145]: Waiting for error group managing segments... Aug 7 21:23:06.901265651 (11h43m28.645322518s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:06.901287666 (11h43m28.645344528s) [upload=12145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:06.901291297 (11h43m28.645348161s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.622244822 (11h43m58.366301669s) [upload=12145]: Done uploading segments... Aug 7 21:23:36.622267267 (11h43m58.366324114s) [upload=12145]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:20.942800102 (11h44m42.686856953s) [upload=12164]: Starting upload Aug 7 21:24:20.942876168 (11h44m42.686933032s) [upload=12164]: Done starting upload Aug 7 21:24:20.942953347 (11h44m42.687010195s) [upload=12164]: Uploading segments... Aug 7 21:24:20.943004925 (11h44m42.687061797s) [upload=12164]: Got next segment. Inline: true Aug 7 21:24:20.943010718 (11h44m42.687067582s) [upload=12164]: Waiting for error group managing segments... Aug 7 21:24:20.943057728 (11h44m42.687114586s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:20.943063424 (11h44m42.687120287s) [upload=12164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:20.943066967 (11h44m42.687123828s) [upload=12164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:50.669999700 (11h45m12.414056561s) [upload=12164]: Done uploading segments... Aug 7 21:24:50.670022022 (11h45m12.414078882s) [upload=12164]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.001461694s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [.................................................XX.........] duration: 282.261741ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:24:50 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 282.290498ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.........................................X..XXXXXXXXXXXXXXXX] duration: 30.001148405s FAIL (2025-08-07 21:24:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001167624s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 852.115864ms FAIL (2025-08-07 21:24:48 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== 68895 (10h54m20.003725754s) [upload=11347]: Done uploading segments... Aug 7 20:33:58.259694049 (10h54m20.00375091s) [upload=11347]: Upload finished. err: <nil> Aug 7 20:34:42.291300370 (10h55m4.035357231s) [upload=11357]: Starting upload Aug 7 20:34:42.291392440 (10h55m4.035449315s) [upload=11357]: Done starting upload Aug 7 20:34:42.291505939 (10h55m4.035562788s) [upload=11357]: Uploading segments... Aug 7 20:34:42.291587798 (10h55m4.035644667s) [upload=11357]: Got next segment. Inline: true Aug 7 20:34:42.291610428 (10h55m4.035667289s) [upload=11357]: Waiting for error group managing segments... Aug 7 20:34:42.291660078 (10h55m4.035716927s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:34:42.291665407 (10h55m4.035722257s) [upload=11357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:34:42.291668818 (10h55m4.035725679s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:34:42.445169337 (10h55m4.189226185s) [upload=11357]: Done uploading segments... Aug 7 20:34:42.445187662 (10h55m4.189244508s) [upload=11357]: Upload finished. err: <nil> Aug 7 20:35:56.993979713 (10h56m18.738036563s) [upload=11378]: Starting upload Aug 7 20:35:56.994042734 (10h56m18.738099594s) [upload=11378]: Done starting upload Aug 7 20:35:56.994133564 (10h56m18.738190418s) [upload=11378]: Uploading segments... Aug 7 20:35:56.994188836 (10h56m18.738245705s) [upload=11378]: Got next segment. Inline: true Aug 7 20:35:56.994194253 (10h56m18.738251113s) [upload=11378]: Waiting for error group managing segments... Aug 7 20:35:56.994255059 (10h56m18.738311921s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:35:56.994261721 (10h56m18.738318586s) [upload=11378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:35:56.994266228 (10h56m18.73832309s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:35:57.196915243 (10h56m18.940972094s) [upload=11378]: Done uploading segments... Aug 7 20:35:57.196933580 (10h56m18.940990431s) [upload=11378]: Upload finished. err: <nil> Aug 7 20:36:48.383926585 (10h57m10.127983431s) [upload=11392]: Starting upload Aug 7 20:36:48.383977279 (10h57m10.128034139s) [upload=11392]: Done starting upload Aug 7 20:36:48.384080928 (10h57m10.128137781s) [upload=11392]: Uploading segments... Aug 7 20:36:48.384124913 (10h57m10.128181781s) [upload=11392]: Got next segment. Inline: true Aug 7 20:36:48.384130188 (10h57m10.128187037s) [upload=11392]: Waiting for error group managing segments... Aug 7 20:36:48.384183298 (10h57m10.128240148s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:36:48.384188890 (10h57m10.128245741s) [upload=11392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:36:48.384192353 (10h57m10.128249203s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:36:48.597716681 (10h57m10.341773532s) [upload=11392]: Done uploading segments... Aug 7 20:36:48.597735626 (10h57m10.341792476s) [upload=11392]: Upload finished. err: <nil> Aug 7 20:37:59.886756865 (10h58m21.630813711s) [upload=11411]: Starting upload Aug 7 20:37:59.886810662 (10h58m21.630867522s) [upload=11411]: Done starting upload Aug 7 20:37:59.886870714 (10h58m21.630927561s) [upload=11411]: Uploading segments... Aug 7 20:37:59.886906763 (10h58m21.630963634s) [upload=11411]: Got next segment. Inline: true Aug 7 20:37:59.886911413 (10h58m21.630968259s) [upload=11411]: Waiting for error group managing segments... Aug 7 20:37:59.886948286 (10h58m21.631005132s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:37:59.886952923 (10h58m21.631009769s) [upload=11411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:37:59.886957250 (10h58m21.631014096s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:38:00.073123067 (10h58m21.817179918s) [upload=11411]: Done uploading segments... Aug 7 20:38:00.073142248 (10h58m21.817199097s) [upload=11411]: Upload finished. err: <nil> Aug 7 20:39:06.192304563 (10h59m27.936361423s) [upload=11429]: Starting upload Aug 7 20:39:06.192379945 (10h59m27.936436804s) [upload=11429]: Done starting upload Aug 7 20:39:06.192450925 (10h59m27.936507772s) [upload=11429]: Uploading segments... Aug 7 20:39:06.192493866 (10h59m27.936550726s) [upload=11429]: Got next segment. Inline: true Aug 7 20:39:06.192499400 (10h59m27.93655626s) [upload=11429]: Waiting for error group managing segments... Aug 7 20:39:06.192549058 (10h59m27.936605918s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:06.192553658 (10h59m27.936610518s) [upload=11429]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:06.192556283 (10h59m27.936613143s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:06.410732058 (10h59m28.154788918s) [upload=11429]: Done uploading segments... Aug 7 20:39:06.410751072 (10h59m28.154807921s) [upload=11429]: Upload finished. err: <nil> Aug 7 20:39:38.189437550 (10h59m59.9334944s) [upload=11438]: Starting upload Aug 7 20:39:38.189494699 (10h59m59.933551565s) [upload=11438]: Done starting upload Aug 7 20:39:38.189570885 (10h59m59.933627735s) [upload=11438]: Uploading segments... Aug 7 20:39:38.189613828 (10h59m59.933670698s) [upload=11438]: Got next segment. Inline: true Aug 7 20:39:38.189618898 (10h59m59.933675758s) [upload=11438]: Waiting for error group managing segments... Aug 7 20:39:38.189671547 (10h59m59.933728396s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:38.189676923 (10h59m59.933733774s) [upload=11438]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:38.189680363 (10h59m59.933737214s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:38.339370287 (11h0m0.083427138s) [upload=11438]: Done uploading segments... Aug 7 20:39:38.339394965 (11h0m0.083451822s) [upload=11438]: Upload finished. err: <nil> Aug 7 20:40:13.049368895 (11h0m34.793425745s) [upload=11448]: Starting upload Aug 7 20:40:13.049412890 (11h0m34.793469743s) [upload=11448]: Done starting upload Aug 7 20:40:13.049490173 (11h0m34.793547027s) [upload=11448]: Uploading segments... Aug 7 20:40:13.049531553 (11h0m34.793588423s) [upload=11448]: Got next segment. Inline: true Aug 7 20:40:13.049536305 (11h0m34.79359316s) [upload=11448]: Waiting for error group managing segments... Aug 7 20:40:13.049606089 (11h0m34.793662937s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:40:13.049611863 (11h0m34.793668715s) [upload=11448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:40:13.049615487 (11h0m34.793672338s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:40:13.251485869 (11h0m34.995542721s) [upload=11448]: Done uploading segments... Aug 7 20:40:13.251504252 (11h0m34.995561102s) [upload=11448]: Upload finished. err: <nil> Aug 7 20:41:01.300323709 (11h1m23.044380561s) [upload=11462]: Starting upload Aug 7 20:41:01.300386685 (11h1m23.044443554s) [upload=11462]: Done starting upload Aug 7 20:41:01.300471424 (11h1m23.044528277s) [upload=11462]: Uploading segments... Aug 7 20:41:01.300528867 (11h1m23.044585736s) [upload=11462]: Got next segment. Inline: true Aug 7 20:41:01.300534426 (11h1m23.044591286s) [upload=11462]: Waiting for error group managing segments... Aug 7 20:41:01.300592418 (11h1m23.044649268s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:01.300598189 (11h1m23.04465505s) [upload=11462]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:01.300601616 (11h1m23.044658476s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:01.448083138 (11h1m23.192139989s) [upload=11462]: Done uploading segments... Aug 7 20:41:01.448102791 (11h1m23.192159644s) [upload=11462]: Upload finished. err: <nil> Aug 7 20:41:25.522931110 (11h1m47.266987978s) [upload=11468]: Starting upload Aug 7 20:41:25.523039051 (11h1m47.267095924s) [upload=11468]: Done starting upload Aug 7 20:41:25.523181759 (11h1m47.267238621s) [upload=11468]: Uploading segments... Aug 7 20:41:25.523403073 (11h1m47.267459944s) [upload=11468]: Got next segment. Inline: true Aug 7 20:41:25.523413059 (11h1m47.26746993s) [upload=11468]: Waiting for error group managing segments... Aug 7 20:41:25.523492519 (11h1m47.267549389s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:25.523498700 (11h1m47.267555566s) [upload=11468]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:25.523502761 (11h1m47.267559629s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:25.840530122 (11h1m47.584586983s) [upload=11468]: Done uploading segments... Aug 7 20:41:25.840544406 (11h1m47.584601253s) [upload=11468]: Upload finished. err: <nil> Aug 7 20:41:49.037046771 (11h2m10.781103621s) [upload=11474]: Starting upload Aug 7 20:41:49.037108376 (11h2m10.781165228s) [upload=11474]: Done starting upload Aug 7 20:41:49.037181045 (11h2m10.781237895s) [upload=11474]: Uploading segments... Aug 7 20:41:49.037238734 (11h2m10.781295606s) [upload=11474]: Got next segment. Inline: true Aug 7 20:41:49.037244163 (11h2m10.781301028s) [upload=11474]: Waiting for error group managing segments... Aug 7 20:41:49.037309765 (11h2m10.781366628s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:49.037314702 (11h2m10.781371562s) [upload=11474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:49.037317447 (11h2m10.781374307s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:49.205260584 (11h2m10.949317434s) [upload=11474]: Done uploading segments... Aug 7 20:41:49.205278299 (11h2m10.94933515s) [upload=11474]: Upload finished. err: <nil> Duration: 852.233072ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 30.001472705s FAIL (2025-08-07 21:24:47 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001498871s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................X......................X............] duration: 2.885155075s FAIL (2025-08-07 21:24:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== oad finished. err: <nil> Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments... Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment... Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment. Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments... Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces. Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments... Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil> Duration: 2.885192593s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 30.001097498s FAIL (2025-08-07 21:24:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001112039s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 1m0.00084478s FAIL (2025-08-07 21:24:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== IMX42Y5QSK4NWA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.722783177 (11h37m53.466840067s) [noise=true , address=104.36.82.18:4385 , piece_id=P55LU5RIEFOXVJMT5ECSZVBJVR3EKTZMTQUYYZADAMWLXQDD4PVA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.723152014 (11h37m53.467208877s) [noise=true , address=79.127.201.215:4354 , piece_id=ZSCV73YKDLJGMUTZC25BKEDN42U4MTK4YXSTQLAL7CJUHGKPC6ZQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.723164484 (11h37m53.467221343s) [noise=true , address=104.36.82.11:4117 , piece_id=WYFZCITVSFFKJWASQ3NJATWLGMNENL3HHJI7UDFARGQKV4NJEOPQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.723613232 (11h37m53.467670101s) [noise=true , address=104.36.82.58:4283 , piece_id=BGC2FWKKRJACRMUB3W5ZIHLCSHV2GQMIMJQAVLYHTY7O3TLEEOXQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.723920353 (11h37m53.467977215s) [noise=true , address=104.36.82.35:4596 , piece_id=T7IXOFRCQBNKM4KYP26RYXCVJ23BTNQVAWAOS6RJ5FU5PXQNH2VA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.724224848 (11h37m53.468281719s) [noise=true , address=45.91.116.6:4983 , piece_id=SQG4PIM6NRCBNWGNDEKGEAT5NKC5ZPA2AIYMBNVECK2PIXDDK6VA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.724565477 (11h37m53.468622346s) [noise=true , address=104.36.82.52:4114 , piece_id=DOKPKLWNIR5U3DL4H57J2TWFQPCQPS5JYSHXHATQVNCM4MDM3UIQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.724862177 (11h37m53.468919042s) [noise=true , address=104.36.82.62:4410 , piece_id=J2TWQYM2SQC5CYHQTLXDWRX7PF5C2T6ANYJ3NGE3KHZJBJZGR4AQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.725090136 (11h37m53.469146998s) [noise=true , address=104.36.82.61:4386 , piece_id=DHHIO7VLBG4O4J4KZ33DLN7L557WGMKNIOP7M4HTOGKTR5UZWSKA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.725329528 (11h37m53.469386388s) [noise=true , address=45.91.116.5:4307 , piece_id=5U7LO24VJLOCITSG5WNVO7H5XHUP4W254TNSPII6DIBI2BQPMBQA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.725607912 (11h37m53.469664775s) [noise=true , address=104.36.82.36:4618 , piece_id=IDWBF76EB4KMI2U636Q3NJZTY3TLBHWLFC5LAKER6LET7VEUQEDQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.725840955 (11h37m53.469897818s) [noise=true , address=104.36.82.9:4018 , piece_id=OQ4DZQI5SLAWF64WJJD2SWA4NZJG63JCKK6VRW6ZWR2QTX27UKCA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.726098568 (11h37m53.470155431s) [noise=true , address=104.36.82.37:4665 , piece_id=C45VO2IWK4LW4J23K4BJT6KQT3DMXKC5NZ4H4H7TD3V4WEKCUASQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.726341913 (11h37m53.470398777s) [noise=true , address=104.36.82.59:4301 , piece_id=OE2HM5UBT5G7SAELRRZVDHMNSVTC3SF6PIOOOELWXFRYGD4TESYA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.726645507 (11h37m53.470702371s) [noise=true , address=79.127.219.47:4058 , piece_id=7OIUVNDUPCYRK5CCQBI6KOBMYRKCKHPRGIVC5PYKDDW3ZIUOBIFQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.726938117 (11h37m53.470994979s) [noise=true , address=45.91.116.14:4200 , piece_id=LTYXWK7RBFXWUCQOSQPQC3YZSWD55CXGL76SNSQRJCBO7HEJLRXQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:32.333624838 (11h37m54.077681691s) [upload=12056]: Next returned nil segment Aug 7 21:17:32.333642519 (11h37m54.07769937s) [upload=12056]: Waiting for error group managing segments... Aug 7 21:17:32.499764814 (11h37m54.243821675s) [noise=true , address=104.36.82.35:4596 , piece_id=T7IXOFRCQBNKM4KYP26RYXCVJ23BTNQVAWAOS6RJ5FU5PXQNH2VA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.500372993 (11h37m54.244429854s) [noise=true , address=79.127.219.47:4058 , piece_id=7OIUVNDUPCYRK5CCQBI6KOBMYRKCKHPRGIVC5PYKDDW3ZIUOBIFQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.527953393 (11h37m54.272010288s) [noise=true , address=104.36.82.38:4700 , piece_id=RKXSWNPNDE77WSUTVE5ORSPLF2CP73FWMTKYNH2WETS7KVRPDYRQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.528587479 (11h37m54.272644333s) [noise=true , address=104.36.82.36:4618 , piece_id=IDWBF76EB4KMI2U636Q3NJZTY3TLBHWLFC5LAKER6LET7VEUQEDQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.532068888 (11h37m54.276125739s) [noise=true , address=104.36.82.58:4283 , piece_id=BGC2FWKKRJACRMUB3W5ZIHLCSHV2GQMIMJQAVLYHTY7O3TLEEOXQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.535806115 (11h37m54.279862977s) [noise=true , address=104.36.82.52:4114 , piece_id=DOKPKLWNIR5U3DL4H57J2TWFQPCQPS5JYSHXHATQVNCM4MDM3UIQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.549937686 (11h37m54.293994536s) [noise=true , address=104.36.82.11:4117 , piece_id=WYFZCITVSFFKJWASQ3NJATWLGMNENL3HHJI7UDFARGQKV4NJEOPQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.578063947 (11h37m54.322120801s) [noise=true , address=45.91.116.14:4200 , piece_id=LTYXWK7RBFXWUCQOSQPQC3YZSWD55CXGL76SNSQRJCBO7HEJLRXQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.584491919 (11h37m54.328548767s) [noise=true , address=104.36.82.61:4386 , piece_id=DHHIO7VLBG4O4J4KZ33DLN7L557WGMKNIOP7M4HTOGKTR5UZWSKA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.589474263 (11h37m54.333531109s) [noise=true , address=104.36.82.62:4410 , piece_id=J2TWQYM2SQC5CYHQTLXDWRX7PF5C2T6ANYJ3NGE3KHZJBJZGR4AQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.599981486 (11h37m54.344038336s) [noise=true , address=104.36.82.59:4301 , piece_id=OE2HM5UBT5G7SAELRRZVDHMNSVTC3SF6PIOOOELWXFRYGD4TESYA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.627542635 (11h37m54.371599485s) [noise=true , address=45.91.116.9:4286 , piece_id=K6JVV2LJR3624665CRKJRFPFK3W2SUTL3SHPJPIMX42Y5QSK4NWA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.695607895 (11h37m54.439664748s) [noise=true , address=45.91.116.5:4307 , piece_id=5U7LO24VJLOCITSG5WNVO7H5XHUP4W254TNSPII6DIBI2BQPMBQA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.697853180 (11h37m54.44191004s) [noise=true , address=104.36.82.37:4665 , piece_id=C45VO2IWK4LW4J23K4BJT6KQT3DMXKC5NZ4H4H7TD3V4WEKCUASQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.697876596 (11h37m54.441933447s) [seg_pos={0 0}, upload=12056]: Segment reached optimal threshold of 14 pieces. Aug 7 21:17:32.698050022 (11h37m54.442106873s) [noise=true , address=45.91.116.6:4983 , piece_id=SQG4PIM6NRCBNWGNDEKGEAT5NKC5ZPA2AIYMBNVECK2PIXDDK6VA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122trJbDHuqeBMQHZT97HP2PgkeuLyUiDiuRm4nayu6ZTH7tat6): protocol: expected piece hash; context canceled; EOF Aug 7 21:17:32.698076888 (11h37m54.442133747s) [noise=true , address=79.127.201.215:4354 , piece_id=ZSCV73YKDLJGMUTZC25BKEDN42U4MTK4YXSTQLAL7CJUHGKPC6ZQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13Jtv8kL5R9oVro2D4ioqsznoJBXyNgJLdj1FcrGN3WWH45rkP): protocol: expected piece hash; context canceled; EOF Aug 7 21:17:32.698154719 (11h37m54.442211568s) [noise=true , address=104.36.82.18:4385 , piece_id=P55LU5RIEFOXVJMT5ECSZVBJVR3EKTZMTQUYYZADAMWLXQDD4PVA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123PZh1ieWYoc9U5CRrRzV1ZTpGYmjLmh5CpRy1NRC8wiLkumbs): protocol: expected piece hash; context canceled; EOF Aug 7 21:17:32.698262049 (11h37m54.442318898s) [noise=true , address=104.36.82.9:4018 , piece_id=OQ4DZQI5SLAWF64WJJD2SWA4NZJG63JCKK6VRW6ZWR2QTX27UKCA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFRHbXjTaxoEKc5gbQWYt6jwUg8jknScBZpc8PdKNSZhYqUqP): protocol: write tcp 139.59.144.22:47050->104.36.82.9:4018: use of closed network connection; write tcp 139.59.144.22:47050->104.36.82.9:4018: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47050->104.36.82.9:4018: use of closed network connection Aug 7 21:17:32.698294367 (11h37m54.442351219s) [seg_pos={0 0}, upload=12056]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:17:32.698397017 (11h37m54.442453885s) [upload=12056]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:17:32.698404035 (11h37m54.442460903s) [upload=12056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:34.247964577 (11h43m55.99202143s) [upload=12056]: Done uploading segments... Aug 7 21:23:34.247989370 (11h43m55.992046231s) [upload=12056]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:34.651573101 (11h43m56.395629952s) [upload=12153]: Starting upload Aug 7 21:23:34.651679674 (11h43m56.395736523s) [upload=12153]: Done starting upload Aug 7 21:23:34.651758369 (11h43m56.395815219s) [upload=12153]: Uploading segments... Aug 7 21:23:34.651939812 (11h43m56.395996659s) [upload=12153]: Got next segment. Inline: false Aug 7 21:23:34.651977372 (11h43m56.396034218s) [upload=12153]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:34.651981823 (11h43m56.396038683s) [upload=12153]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:24:34.381844223 (11h44m56.125901093s) [upload=12153]: Next segment err: context deadline exceeded Aug 7 21:24:34.382111867 (11h44m56.126168721s) [upload=12153]: Done uploading segments... Aug 7 21:24:34.382134105 (11h44m56.126190955s) [upload=12153]: Upload finished. err: context deadline exceeded Duration: 1m0.000858668s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 30.001430252s FAIL (2025-08-07 21:24:21 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001465185s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [..........................................X..XXXXXXXXXXXXXXX] duration: 30.002046871s FAIL (2025-08-07 21:24:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.002073419s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................X......................X...........] duration: 2.830722105s FAIL (2025-08-07 21:24:16 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== oad finished. err: <nil> Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments... Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment... Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment. Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments... Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces. Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments... Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil> Duration: 2.830769245s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 9 out of 60 is failed (threshold=3) [..................................................X.XXXXXXXX] duration: 1m0.000936564s FAIL (2025-08-07 21:24:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1.227141194 (11h34m22.971198047s) [upload=12013]: Uploading segments... Aug 7 21:14:01.227219219 (11h34m22.971276084s) [upload=12013]: Got next segment. Inline: true Aug 7 21:14:01.227227170 (11h34m22.971284041s) [upload=12013]: Waiting for error group managing segments... Aug 7 21:14:01.227295298 (11h34m22.971352162s) [upload=12013]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:01.227301580 (11h34m22.971358443s) [upload=12013]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:01.227305300 (11h34m22.971362163s) [upload=12013]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:00.953618154 (11h35m22.697675004s) [upload=12013]: Done uploading segments... Aug 7 21:15:00.953640566 (11h35m22.697697426s) [upload=12013]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:01.394116473 (11h35m23.138173339s) [upload=12030]: Starting upload Aug 7 21:15:01.394242598 (11h35m23.138299459s) [upload=12030]: Done starting upload Aug 7 21:15:01.394373467 (11h35m23.138430318s) [upload=12030]: Uploading segments... Aug 7 21:15:01.394459055 (11h35m23.138515928s) [upload=12030]: Got next segment. Inline: true Aug 7 21:15:01.394467128 (11h35m23.138523996s) [upload=12030]: Waiting for error group managing segments... Aug 7 21:15:01.394548041 (11h35m23.138604915s) [upload=12030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:01.394554816 (11h35m23.13861169s) [upload=12030]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:01.394559518 (11h35m23.138616392s) [upload=12030]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:03.584668408 (11h35m25.328725286s) [upload=12030]: Done uploading segments... Aug 7 21:15:03.584713689 (11h35m25.32877056s) [upload=12030]: Upload finished. err: <nil> Aug 7 21:15:54.636790163 (11h36m16.380847013s) [upload=12044]: Starting upload Aug 7 21:15:54.636877707 (11h36m16.380934638s) [upload=12044]: Done starting upload Aug 7 21:15:54.636936350 (11h36m16.3809932s) [upload=12044]: Uploading segments... Aug 7 21:15:54.636993824 (11h36m16.381050685s) [upload=12044]: Got next segment. Inline: true Aug 7 21:15:54.636999384 (11h36m16.381056244s) [upload=12044]: Waiting for error group managing segments... Aug 7 21:15:54.637044877 (11h36m16.381101727s) [upload=12044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:54.637050662 (11h36m16.381107522s) [upload=12044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:54.637053905 (11h36m16.381110766s) [upload=12044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:54.359175610 (11h37m16.103232476s) [upload=12044]: Done uploading segments... Aug 7 21:16:54.359203156 (11h37m16.103260018s) [upload=12044]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:54.796260076 (11h37m16.54031694s) [upload=12060]: Starting upload Aug 7 21:16:54.796405798 (11h37m16.540462658s) [upload=12060]: Done starting upload Aug 7 21:16:54.796514557 (11h37m16.540571407s) [upload=12060]: Uploading segments... Aug 7 21:16:54.796581012 (11h37m16.540637875s) [upload=12060]: Got next segment. Inline: true Aug 7 21:16:54.796586298 (11h37m16.540643158s) [upload=12060]: Waiting for error group managing segments... Aug 7 21:16:54.796657533 (11h37m16.540714383s) [upload=12060]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:54.796663030 (11h37m16.540719894s) [upload=12060]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:54.796669424 (11h37m16.540726289s) [upload=12060]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:17:54.519327140 (11h38m16.263384004s) [upload=12060]: Done uploading segments... Aug 7 21:17:54.519368022 (11h38m16.263424882s) [upload=12060]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:54.944498267 (11h38m16.688555136s) [upload=12074]: Starting upload Aug 7 21:17:54.944617406 (11h38m16.688674267s) [upload=12074]: Done starting upload Aug 7 21:17:54.944695430 (11h38m16.688752285s) [upload=12074]: Uploading segments... Aug 7 21:17:54.944771865 (11h38m16.688828736s) [upload=12074]: Got next segment. Inline: true Aug 7 21:17:54.944781252 (11h38m16.688838123s) [upload=12074]: Waiting for error group managing segments... Aug 7 21:17:54.944841042 (11h38m16.688897916s) [upload=12074]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:54.944848212 (11h38m16.688905079s) [upload=12074]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:54.944851963 (11h38m16.688908827s) [upload=12074]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:54.673864467 (11h39m16.417921315s) [upload=12074]: Done uploading segments... Aug 7 21:18:54.673885859 (11h39m16.417942708s) [upload=12074]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:55.119912759 (11h39m16.863969619s) [upload=12087]: Starting upload Aug 7 21:18:55.120013170 (11h39m16.864070018s) [upload=12087]: Done starting upload Aug 7 21:18:55.120094735 (11h39m16.864151596s) [upload=12087]: Uploading segments... Aug 7 21:18:55.120142836 (11h39m16.864199705s) [upload=12087]: Got next segment. Inline: true Aug 7 21:18:55.120147776 (11h39m16.864204637s) [upload=12087]: Waiting for error group managing segments... Aug 7 21:18:55.120201712 (11h39m16.864258573s) [upload=12087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:55.120207674 (11h39m16.864264534s) [upload=12087]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:55.120211321 (11h39m16.864268183s) [upload=12087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:54.844368358 (11h40m16.588425206s) [upload=12087]: Done uploading segments... Aug 7 21:19:54.844395070 (11h40m16.588451926s) [upload=12087]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:04.793025148 (11h40m26.537081997s) [upload=12103]: Starting upload Aug 7 21:20:04.793120317 (11h40m26.53717719s) [upload=12103]: Done starting upload Aug 7 21:20:04.793244612 (11h40m26.537301459s) [upload=12103]: Uploading segments... Aug 7 21:20:04.793331764 (11h40m26.537388638s) [upload=12103]: Got next segment. Inline: true Aug 7 21:20:04.793339748 (11h40m26.537396613s) [upload=12103]: Waiting for error group managing segments... Aug 7 21:20:04.793417981 (11h40m26.537474831s) [upload=12103]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:04.793424412 (11h40m26.537481262s) [upload=12103]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:04.793427866 (11h40m26.537484716s) [upload=12103]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:04.512178875 (11h41m26.256235737s) [upload=12103]: Done uploading segments... Aug 7 21:21:04.512205338 (11h41m26.256262189s) [upload=12103]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:14.239199000 (11h41m35.983255849s) [upload=12119]: Starting upload Aug 7 21:21:14.239282320 (11h41m35.983339189s) [upload=12119]: Done starting upload Aug 7 21:21:14.239356308 (11h41m35.98341316s) [upload=12119]: Uploading segments... Aug 7 21:21:14.239528709 (11h41m35.983585615s) [upload=12119]: Got next segment. Inline: true Aug 7 21:21:14.239538735 (11h41m35.983595603s) [upload=12119]: Waiting for error group managing segments... Aug 7 21:21:14.239621427 (11h41m35.983678288s) [upload=12119]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:14.239629030 (11h41m35.983685885s) [upload=12119]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:14.239632488 (11h41m35.98368934s) [upload=12119]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:13.962302553 (11h42m35.706359403s) [upload=12119]: Done uploading segments... Aug 7 21:22:13.962327972 (11h42m35.706384822s) [upload=12119]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:14.360206935 (11h42m36.104263785s) [upload=12134]: Starting upload Aug 7 21:22:14.360271125 (11h42m36.104327976s) [upload=12134]: Done starting upload Aug 7 21:22:14.360336747 (11h42m36.104393607s) [upload=12134]: Uploading segments... Aug 7 21:22:14.360380199 (11h42m36.104437068s) [upload=12134]: Got next segment. Inline: true Aug 7 21:22:14.360385659 (11h42m36.10444252s) [upload=12134]: Waiting for error group managing segments... Aug 7 21:22:14.360431265 (11h42m36.104488116s) [upload=12134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:14.360437033 (11h42m36.104493884s) [upload=12134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:14.360440534 (11h42m36.104497395s) [upload=12134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:14.088328715 (11h43m35.832385565s) [upload=12134]: Done uploading segments... Aug 7 21:23:14.088350462 (11h43m35.832407323s) [upload=12134]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:14.505772185 (11h43m36.249829051s) [upload=12148]: Starting upload Aug 7 21:23:14.505877531 (11h43m36.249934403s) [upload=12148]: Done starting upload Aug 7 21:23:14.505973975 (11h43m36.250030823s) [upload=12148]: Uploading segments... Aug 7 21:23:14.506132768 (11h43m36.250189638s) [upload=12148]: Got next segment. Inline: true Aug 7 21:23:14.506140538 (11h43m36.250197403s) [upload=12148]: Waiting for error group managing segments... Aug 7 21:23:14.506211738 (11h43m36.250268603s) [upload=12148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:14.506218667 (11h43m36.250275531s) [upload=12148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:14.506222979 (11h43m36.250279842s) [upload=12148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:14.230441064 (11h44m35.974497931s) [upload=12148]: Done uploading segments... Aug 7 21:24:14.230468679 (11h44m35.974525551s) [upload=12148]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.000952955s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 846.488155ms FAIL (2025-08-07 21:24:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== 68895 (10h54m20.003725754s) [upload=11347]: Done uploading segments... Aug 7 20:33:58.259694049 (10h54m20.00375091s) [upload=11347]: Upload finished. err: <nil> Aug 7 20:34:42.291300370 (10h55m4.035357231s) [upload=11357]: Starting upload Aug 7 20:34:42.291392440 (10h55m4.035449315s) [upload=11357]: Done starting upload Aug 7 20:34:42.291505939 (10h55m4.035562788s) [upload=11357]: Uploading segments... Aug 7 20:34:42.291587798 (10h55m4.035644667s) [upload=11357]: Got next segment. Inline: true Aug 7 20:34:42.291610428 (10h55m4.035667289s) [upload=11357]: Waiting for error group managing segments... Aug 7 20:34:42.291660078 (10h55m4.035716927s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:34:42.291665407 (10h55m4.035722257s) [upload=11357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:34:42.291668818 (10h55m4.035725679s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:34:42.445169337 (10h55m4.189226185s) [upload=11357]: Done uploading segments... Aug 7 20:34:42.445187662 (10h55m4.189244508s) [upload=11357]: Upload finished. err: <nil> Aug 7 20:35:56.993979713 (10h56m18.738036563s) [upload=11378]: Starting upload Aug 7 20:35:56.994042734 (10h56m18.738099594s) [upload=11378]: Done starting upload Aug 7 20:35:56.994133564 (10h56m18.738190418s) [upload=11378]: Uploading segments... Aug 7 20:35:56.994188836 (10h56m18.738245705s) [upload=11378]: Got next segment. Inline: true Aug 7 20:35:56.994194253 (10h56m18.738251113s) [upload=11378]: Waiting for error group managing segments... Aug 7 20:35:56.994255059 (10h56m18.738311921s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:35:56.994261721 (10h56m18.738318586s) [upload=11378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:35:56.994266228 (10h56m18.73832309s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:35:57.196915243 (10h56m18.940972094s) [upload=11378]: Done uploading segments... Aug 7 20:35:57.196933580 (10h56m18.940990431s) [upload=11378]: Upload finished. err: <nil> Aug 7 20:36:48.383926585 (10h57m10.127983431s) [upload=11392]: Starting upload Aug 7 20:36:48.383977279 (10h57m10.128034139s) [upload=11392]: Done starting upload Aug 7 20:36:48.384080928 (10h57m10.128137781s) [upload=11392]: Uploading segments... Aug 7 20:36:48.384124913 (10h57m10.128181781s) [upload=11392]: Got next segment. Inline: true Aug 7 20:36:48.384130188 (10h57m10.128187037s) [upload=11392]: Waiting for error group managing segments... Aug 7 20:36:48.384183298 (10h57m10.128240148s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:36:48.384188890 (10h57m10.128245741s) [upload=11392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:36:48.384192353 (10h57m10.128249203s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:36:48.597716681 (10h57m10.341773532s) [upload=11392]: Done uploading segments... Aug 7 20:36:48.597735626 (10h57m10.341792476s) [upload=11392]: Upload finished. err: <nil> Aug 7 20:37:59.886756865 (10h58m21.630813711s) [upload=11411]: Starting upload Aug 7 20:37:59.886810662 (10h58m21.630867522s) [upload=11411]: Done starting upload Aug 7 20:37:59.886870714 (10h58m21.630927561s) [upload=11411]: Uploading segments... Aug 7 20:37:59.886906763 (10h58m21.630963634s) [upload=11411]: Got next segment. Inline: true Aug 7 20:37:59.886911413 (10h58m21.630968259s) [upload=11411]: Waiting for error group managing segments... Aug 7 20:37:59.886948286 (10h58m21.631005132s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:37:59.886952923 (10h58m21.631009769s) [upload=11411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:37:59.886957250 (10h58m21.631014096s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:38:00.073123067 (10h58m21.817179918s) [upload=11411]: Done uploading segments... Aug 7 20:38:00.073142248 (10h58m21.817199097s) [upload=11411]: Upload finished. err: <nil> Aug 7 20:39:06.192304563 (10h59m27.936361423s) [upload=11429]: Starting upload Aug 7 20:39:06.192379945 (10h59m27.936436804s) [upload=11429]: Done starting upload Aug 7 20:39:06.192450925 (10h59m27.936507772s) [upload=11429]: Uploading segments... Aug 7 20:39:06.192493866 (10h59m27.936550726s) [upload=11429]: Got next segment. Inline: true Aug 7 20:39:06.192499400 (10h59m27.93655626s) [upload=11429]: Waiting for error group managing segments... Aug 7 20:39:06.192549058 (10h59m27.936605918s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:06.192553658 (10h59m27.936610518s) [upload=11429]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:06.192556283 (10h59m27.936613143s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:06.410732058 (10h59m28.154788918s) [upload=11429]: Done uploading segments... Aug 7 20:39:06.410751072 (10h59m28.154807921s) [upload=11429]: Upload finished. err: <nil> Aug 7 20:39:38.189437550 (10h59m59.9334944s) [upload=11438]: Starting upload Aug 7 20:39:38.189494699 (10h59m59.933551565s) [upload=11438]: Done starting upload Aug 7 20:39:38.189570885 (10h59m59.933627735s) [upload=11438]: Uploading segments... Aug 7 20:39:38.189613828 (10h59m59.933670698s) [upload=11438]: Got next segment. Inline: true Aug 7 20:39:38.189618898 (10h59m59.933675758s) [upload=11438]: Waiting for error group managing segments... Aug 7 20:39:38.189671547 (10h59m59.933728396s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:38.189676923 (10h59m59.933733774s) [upload=11438]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:38.189680363 (10h59m59.933737214s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:38.339370287 (11h0m0.083427138s) [upload=11438]: Done uploading segments... Aug 7 20:39:38.339394965 (11h0m0.083451822s) [upload=11438]: Upload finished. err: <nil> Aug 7 20:40:13.049368895 (11h0m34.793425745s) [upload=11448]: Starting upload Aug 7 20:40:13.049412890 (11h0m34.793469743s) [upload=11448]: Done starting upload Aug 7 20:40:13.049490173 (11h0m34.793547027s) [upload=11448]: Uploading segments... Aug 7 20:40:13.049531553 (11h0m34.793588423s) [upload=11448]: Got next segment. Inline: true Aug 7 20:40:13.049536305 (11h0m34.79359316s) [upload=11448]: Waiting for error group managing segments... Aug 7 20:40:13.049606089 (11h0m34.793662937s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:40:13.049611863 (11h0m34.793668715s) [upload=11448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:40:13.049615487 (11h0m34.793672338s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:40:13.251485869 (11h0m34.995542721s) [upload=11448]: Done uploading segments... Aug 7 20:40:13.251504252 (11h0m34.995561102s) [upload=11448]: Upload finished. err: <nil> Aug 7 20:41:01.300323709 (11h1m23.044380561s) [upload=11462]: Starting upload Aug 7 20:41:01.300386685 (11h1m23.044443554s) [upload=11462]: Done starting upload Aug 7 20:41:01.300471424 (11h1m23.044528277s) [upload=11462]: Uploading segments... Aug 7 20:41:01.300528867 (11h1m23.044585736s) [upload=11462]: Got next segment. Inline: true Aug 7 20:41:01.300534426 (11h1m23.044591286s) [upload=11462]: Waiting for error group managing segments... Aug 7 20:41:01.300592418 (11h1m23.044649268s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:01.300598189 (11h1m23.04465505s) [upload=11462]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:01.300601616 (11h1m23.044658476s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:01.448083138 (11h1m23.192139989s) [upload=11462]: Done uploading segments... Aug 7 20:41:01.448102791 (11h1m23.192159644s) [upload=11462]: Upload finished. err: <nil> Aug 7 20:41:25.522931110 (11h1m47.266987978s) [upload=11468]: Starting upload Aug 7 20:41:25.523039051 (11h1m47.267095924s) [upload=11468]: Done starting upload Aug 7 20:41:25.523181759 (11h1m47.267238621s) [upload=11468]: Uploading segments... Aug 7 20:41:25.523403073 (11h1m47.267459944s) [upload=11468]: Got next segment. Inline: true Aug 7 20:41:25.523413059 (11h1m47.26746993s) [upload=11468]: Waiting for error group managing segments... Aug 7 20:41:25.523492519 (11h1m47.267549389s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:25.523498700 (11h1m47.267555566s) [upload=11468]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:25.523502761 (11h1m47.267559629s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:25.840530122 (11h1m47.584586983s) [upload=11468]: Done uploading segments... Aug 7 20:41:25.840544406 (11h1m47.584601253s) [upload=11468]: Upload finished. err: <nil> Aug 7 20:41:49.037046771 (11h2m10.781103621s) [upload=11474]: Starting upload Aug 7 20:41:49.037108376 (11h2m10.781165228s) [upload=11474]: Done starting upload Aug 7 20:41:49.037181045 (11h2m10.781237895s) [upload=11474]: Uploading segments... Aug 7 20:41:49.037238734 (11h2m10.781295606s) [upload=11474]: Got next segment. Inline: true Aug 7 20:41:49.037244163 (11h2m10.781301028s) [upload=11474]: Waiting for error group managing segments... Aug 7 20:41:49.037309765 (11h2m10.781366628s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:49.037314702 (11h2m10.781371562s) [upload=11474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:49.037317447 (11h2m10.781374307s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:49.205260584 (11h2m10.949317434s) [upload=11474]: Done uploading segments... Aug 7 20:41:49.205278299 (11h2m10.94933515s) [upload=11474]: Upload finished. err: <nil> Duration: 846.516269ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [..................................................XX........] duration: 331.809199ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:24:13 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 331.838379ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 298.07836ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:24:12 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 298.102962ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [..................................................XXXXXXXXXX] duration: 1m0.000898158s FAIL (2025-08-07 21:24:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000926799s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 30.000939573s FAIL (2025-08-07 21:24:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000965891s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 30.001297775s FAIL (2025-08-07 21:23:37 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.00132394s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 15 out of 60 is failed (threshold=3) [............................................XX.XXXXXXXXXXXXX] duration: 30.000699788s FAIL (2025-08-07 21:23:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== m33.05890295s) [upload=12064]: Uploading segments... Aug 7 21:17:11.314935220 (11h37m33.058992089s) [upload=12064]: Got next segment. Inline: true Aug 7 21:17:11.314940559 (11h37m33.058997419s) [upload=12064]: Waiting for error group managing segments... Aug 7 21:17:11.314984274 (11h37m33.059041125s) [upload=12064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:11.314989343 (11h37m33.059046203s) [upload=12064]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:11.314992536 (11h37m33.059049396s) [upload=12064]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:17:41.043046082 (11h38m2.787102951s) [upload=12064]: Done uploading segments... Aug 7 21:17:41.043076101 (11h38m2.787132966s) [upload=12064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:41.498964594 (11h38m3.243021445s) [upload=12072]: Starting upload Aug 7 21:17:41.499038639 (11h38m3.243095509s) [upload=12072]: Done starting upload Aug 7 21:17:41.499118466 (11h38m3.243175318s) [upload=12072]: Uploading segments... Aug 7 21:17:41.499176424 (11h38m3.243233293s) [upload=12072]: Got next segment. Inline: true Aug 7 21:17:41.499181917 (11h38m3.243238777s) [upload=12072]: Waiting for error group managing segments... Aug 7 21:17:41.499236606 (11h38m3.243293472s) [upload=12072]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:41.499242465 (11h38m3.24329933s) [upload=12072]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:41.499245854 (11h38m3.243302721s) [upload=12072]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:11.231025141 (11h38m32.97508201s) [upload=12072]: Done uploading segments... Aug 7 21:18:11.231066796 (11h38m32.975123655s) [upload=12072]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:11.675991066 (11h38m33.420047932s) [upload=12078]: Starting upload Aug 7 21:18:11.676081621 (11h38m33.42013848s) [upload=12078]: Done starting upload Aug 7 21:18:11.676172293 (11h38m33.420229154s) [upload=12078]: Uploading segments... Aug 7 21:18:11.676235479 (11h38m33.420292347s) [upload=12078]: Got next segment. Inline: true Aug 7 21:18:11.676241644 (11h38m33.420298508s) [upload=12078]: Waiting for error group managing segments... Aug 7 21:18:11.676304521 (11h38m33.420361386s) [upload=12078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:11.676310489 (11h38m33.420367352s) [upload=12078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:11.676314037 (11h38m33.420370906s) [upload=12078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:41.388891420 (11h39m3.132948271s) [upload=12078]: Done uploading segments... Aug 7 21:18:41.388921897 (11h39m3.132978749s) [upload=12078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:53.498741452 (11h39m15.242798302s) [upload=12086]: Starting upload Aug 7 21:18:53.498798950 (11h39m15.242855819s) [upload=12086]: Done starting upload Aug 7 21:18:53.498860950 (11h39m15.242917802s) [upload=12086]: Uploading segments... Aug 7 21:18:53.498901325 (11h39m15.242958195s) [upload=12086]: Got next segment. Inline: true Aug 7 21:18:53.498908406 (11h39m15.242965267s) [upload=12086]: Waiting for error group managing segments... Aug 7 21:18:53.498951359 (11h39m15.24300821s) [upload=12086]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:53.498956808 (11h39m15.243013668s) [upload=12086]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:53.498960205 (11h39m15.243017065s) [upload=12086]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:23.226932815 (11h39m44.970989679s) [upload=12086]: Done uploading segments... Aug 7 21:19:23.226954988 (11h39m44.971011848s) [upload=12086]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:19:23.633524671 (11h39m45.37758152s) [upload=12093]: Starting upload Aug 7 21:19:23.633599963 (11h39m45.377656832s) [upload=12093]: Done starting upload Aug 7 21:19:23.633677463 (11h39m45.377734313s) [upload=12093]: Uploading segments... Aug 7 21:19:23.633732206 (11h39m45.377789075s) [upload=12093]: Got next segment. Inline: true Aug 7 21:19:23.633737928 (11h39m45.377794778s) [upload=12093]: Waiting for error group managing segments... Aug 7 21:19:23.633790184 (11h39m45.377847035s) [upload=12093]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:19:23.633795686 (11h39m45.377852546s) [upload=12093]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:19:23.633801264 (11h39m45.377858124s) [upload=12093]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:53.363781250 (11h40m15.107838111s) [upload=12093]: Done uploading segments... Aug 7 21:19:53.363801573 (11h40m15.107858434s) [upload=12093]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:24.226894126 (11h40m45.970950987s) [upload=12106]: Starting upload Aug 7 21:20:24.226954589 (11h40m45.971011441s) [upload=12106]: Done starting upload Aug 7 21:20:24.227046095 (11h40m45.971102945s) [upload=12106]: Uploading segments... Aug 7 21:20:24.227086861 (11h40m45.97114373s) [upload=12106]: Got next segment. Inline: true Aug 7 21:20:24.227092422 (11h40m45.971149283s) [upload=12106]: Waiting for error group managing segments... Aug 7 21:20:24.227144188 (11h40m45.971201039s) [upload=12106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:24.227149758 (11h40m45.971206619s) [upload=12106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:24.227153486 (11h40m45.971210345s) [upload=12106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:53.942184977 (11h41m15.68624183s) [upload=12106]: Done uploading segments... Aug 7 21:20:53.942206917 (11h41m15.686263767s) [upload=12106]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:03.619018325 (11h41m25.363075197s) [upload=12116]: Starting upload Aug 7 21:21:03.619120575 (11h41m25.363177439s) [upload=12116]: Done starting upload Aug 7 21:21:03.619209054 (11h41m25.363265908s) [upload=12116]: Uploading segments... Aug 7 21:21:03.619303607 (11h41m25.363360476s) [upload=12116]: Got next segment. Inline: true Aug 7 21:21:03.619310139 (11h41m25.363366989s) [upload=12116]: Waiting for error group managing segments... Aug 7 21:21:03.619383781 (11h41m25.363440632s) [upload=12116]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:03.619390203 (11h41m25.363447055s) [upload=12116]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:03.619504810 (11h41m25.363561678s) [upload=12116]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:33.340185414 (11h41m55.084242267s) [upload=12116]: Done uploading segments... Aug 7 21:21:33.340212992 (11h41m55.08426986s) [upload=12116]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:33.783052975 (11h41m55.527109844s) [upload=12126]: Starting upload Aug 7 21:21:33.783141523 (11h41m55.527198395s) [upload=12126]: Done starting upload Aug 7 21:21:33.783231997 (11h41m55.527288844s) [upload=12126]: Uploading segments... Aug 7 21:21:33.783290191 (11h41m55.527347062s) [upload=12126]: Got next segment. Inline: true Aug 7 21:21:33.783297437 (11h41m55.527354312s) [upload=12126]: Waiting for error group managing segments... Aug 7 21:21:33.783352192 (11h41m55.527409054s) [upload=12126]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:33.783359317 (11h41m55.527416182s) [upload=12126]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:33.783363849 (11h41m55.527420713s) [upload=12126]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:03.508585017 (11h42m25.252641871s) [upload=12126]: Done uploading segments... Aug 7 21:22:03.508625142 (11h42m25.252681994s) [upload=12126]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:03.918704824 (11h42m25.662761676s) [upload=12132]: Starting upload Aug 7 21:22:03.918771848 (11h42m25.662828712s) [upload=12132]: Done starting upload Aug 7 21:22:03.918836144 (11h42m25.662892995s) [upload=12132]: Uploading segments... Aug 7 21:22:03.918882565 (11h42m25.662939434s) [upload=12132]: Got next segment. Inline: true Aug 7 21:22:03.918887612 (11h42m25.662944465s) [upload=12132]: Waiting for error group managing segments... Aug 7 21:22:03.918934357 (11h42m25.662991223s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:03.918943101 (11h42m25.662999966s) [upload=12132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:03.918952300 (11h42m25.663009183s) [upload=12132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:33.649567967 (11h42m55.393624817s) [upload=12132]: Done uploading segments... Aug 7 21:22:33.649593937 (11h42m55.3936508s) [upload=12132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:06.900544850 (11h43m28.644601713s) [upload=12145]: Starting upload Aug 7 21:23:06.900612638 (11h43m28.644669489s) [upload=12145]: Done starting upload Aug 7 21:23:06.901077845 (11h43m28.645134714s) [upload=12145]: Uploading segments... Aug 7 21:23:06.901186098 (11h43m28.645242968s) [upload=12145]: Got next segment. Inline: true Aug 7 21:23:06.901209675 (11h43m28.645266536s) [upload=12145]: Waiting for error group managing segments... Aug 7 21:23:06.901265651 (11h43m28.645322518s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:06.901287666 (11h43m28.645344528s) [upload=12145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:06.901291297 (11h43m28.645348161s) [upload=12145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:36.622244822 (11h43m58.366301669s) [upload=12145]: Done uploading segments... Aug 7 21:23:36.622267267 (11h43m58.366324114s) [upload=12145]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000728401s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X.......X...................................................] duration: 857.461936ms FAIL (2025-08-07 21:23:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== 68895 (10h54m20.003725754s) [upload=11347]: Done uploading segments... Aug 7 20:33:58.259694049 (10h54m20.00375091s) [upload=11347]: Upload finished. err: <nil> Aug 7 20:34:42.291300370 (10h55m4.035357231s) [upload=11357]: Starting upload Aug 7 20:34:42.291392440 (10h55m4.035449315s) [upload=11357]: Done starting upload Aug 7 20:34:42.291505939 (10h55m4.035562788s) [upload=11357]: Uploading segments... Aug 7 20:34:42.291587798 (10h55m4.035644667s) [upload=11357]: Got next segment. Inline: true Aug 7 20:34:42.291610428 (10h55m4.035667289s) [upload=11357]: Waiting for error group managing segments... Aug 7 20:34:42.291660078 (10h55m4.035716927s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:34:42.291665407 (10h55m4.035722257s) [upload=11357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:34:42.291668818 (10h55m4.035725679s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:34:42.445169337 (10h55m4.189226185s) [upload=11357]: Done uploading segments... Aug 7 20:34:42.445187662 (10h55m4.189244508s) [upload=11357]: Upload finished. err: <nil> Aug 7 20:35:56.993979713 (10h56m18.738036563s) [upload=11378]: Starting upload Aug 7 20:35:56.994042734 (10h56m18.738099594s) [upload=11378]: Done starting upload Aug 7 20:35:56.994133564 (10h56m18.738190418s) [upload=11378]: Uploading segments... Aug 7 20:35:56.994188836 (10h56m18.738245705s) [upload=11378]: Got next segment. Inline: true Aug 7 20:35:56.994194253 (10h56m18.738251113s) [upload=11378]: Waiting for error group managing segments... Aug 7 20:35:56.994255059 (10h56m18.738311921s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:35:56.994261721 (10h56m18.738318586s) [upload=11378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:35:56.994266228 (10h56m18.73832309s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:35:57.196915243 (10h56m18.940972094s) [upload=11378]: Done uploading segments... Aug 7 20:35:57.196933580 (10h56m18.940990431s) [upload=11378]: Upload finished. err: <nil> Aug 7 20:36:48.383926585 (10h57m10.127983431s) [upload=11392]: Starting upload Aug 7 20:36:48.383977279 (10h57m10.128034139s) [upload=11392]: Done starting upload Aug 7 20:36:48.384080928 (10h57m10.128137781s) [upload=11392]: Uploading segments... Aug 7 20:36:48.384124913 (10h57m10.128181781s) [upload=11392]: Got next segment. Inline: true Aug 7 20:36:48.384130188 (10h57m10.128187037s) [upload=11392]: Waiting for error group managing segments... Aug 7 20:36:48.384183298 (10h57m10.128240148s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:36:48.384188890 (10h57m10.128245741s) [upload=11392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:36:48.384192353 (10h57m10.128249203s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:36:48.597716681 (10h57m10.341773532s) [upload=11392]: Done uploading segments... Aug 7 20:36:48.597735626 (10h57m10.341792476s) [upload=11392]: Upload finished. err: <nil> Aug 7 20:37:59.886756865 (10h58m21.630813711s) [upload=11411]: Starting upload Aug 7 20:37:59.886810662 (10h58m21.630867522s) [upload=11411]: Done starting upload Aug 7 20:37:59.886870714 (10h58m21.630927561s) [upload=11411]: Uploading segments... Aug 7 20:37:59.886906763 (10h58m21.630963634s) [upload=11411]: Got next segment. Inline: true Aug 7 20:37:59.886911413 (10h58m21.630968259s) [upload=11411]: Waiting for error group managing segments... Aug 7 20:37:59.886948286 (10h58m21.631005132s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:37:59.886952923 (10h58m21.631009769s) [upload=11411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:37:59.886957250 (10h58m21.631014096s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:38:00.073123067 (10h58m21.817179918s) [upload=11411]: Done uploading segments... Aug 7 20:38:00.073142248 (10h58m21.817199097s) [upload=11411]: Upload finished. err: <nil> Aug 7 20:39:06.192304563 (10h59m27.936361423s) [upload=11429]: Starting upload Aug 7 20:39:06.192379945 (10h59m27.936436804s) [upload=11429]: Done starting upload Aug 7 20:39:06.192450925 (10h59m27.936507772s) [upload=11429]: Uploading segments... Aug 7 20:39:06.192493866 (10h59m27.936550726s) [upload=11429]: Got next segment. Inline: true Aug 7 20:39:06.192499400 (10h59m27.93655626s) [upload=11429]: Waiting for error group managing segments... Aug 7 20:39:06.192549058 (10h59m27.936605918s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:06.192553658 (10h59m27.936610518s) [upload=11429]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:06.192556283 (10h59m27.936613143s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:06.410732058 (10h59m28.154788918s) [upload=11429]: Done uploading segments... Aug 7 20:39:06.410751072 (10h59m28.154807921s) [upload=11429]: Upload finished. err: <nil> Aug 7 20:39:38.189437550 (10h59m59.9334944s) [upload=11438]: Starting upload Aug 7 20:39:38.189494699 (10h59m59.933551565s) [upload=11438]: Done starting upload Aug 7 20:39:38.189570885 (10h59m59.933627735s) [upload=11438]: Uploading segments... Aug 7 20:39:38.189613828 (10h59m59.933670698s) [upload=11438]: Got next segment. Inline: true Aug 7 20:39:38.189618898 (10h59m59.933675758s) [upload=11438]: Waiting for error group managing segments... Aug 7 20:39:38.189671547 (10h59m59.933728396s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:38.189676923 (10h59m59.933733774s) [upload=11438]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:38.189680363 (10h59m59.933737214s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:38.339370287 (11h0m0.083427138s) [upload=11438]: Done uploading segments... Aug 7 20:39:38.339394965 (11h0m0.083451822s) [upload=11438]: Upload finished. err: <nil> Aug 7 20:40:13.049368895 (11h0m34.793425745s) [upload=11448]: Starting upload Aug 7 20:40:13.049412890 (11h0m34.793469743s) [upload=11448]: Done starting upload Aug 7 20:40:13.049490173 (11h0m34.793547027s) [upload=11448]: Uploading segments... Aug 7 20:40:13.049531553 (11h0m34.793588423s) [upload=11448]: Got next segment. Inline: true Aug 7 20:40:13.049536305 (11h0m34.79359316s) [upload=11448]: Waiting for error group managing segments... Aug 7 20:40:13.049606089 (11h0m34.793662937s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:40:13.049611863 (11h0m34.793668715s) [upload=11448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:40:13.049615487 (11h0m34.793672338s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:40:13.251485869 (11h0m34.995542721s) [upload=11448]: Done uploading segments... Aug 7 20:40:13.251504252 (11h0m34.995561102s) [upload=11448]: Upload finished. err: <nil> Aug 7 20:41:01.300323709 (11h1m23.044380561s) [upload=11462]: Starting upload Aug 7 20:41:01.300386685 (11h1m23.044443554s) [upload=11462]: Done starting upload Aug 7 20:41:01.300471424 (11h1m23.044528277s) [upload=11462]: Uploading segments... Aug 7 20:41:01.300528867 (11h1m23.044585736s) [upload=11462]: Got next segment. Inline: true Aug 7 20:41:01.300534426 (11h1m23.044591286s) [upload=11462]: Waiting for error group managing segments... Aug 7 20:41:01.300592418 (11h1m23.044649268s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:01.300598189 (11h1m23.04465505s) [upload=11462]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:01.300601616 (11h1m23.044658476s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:01.448083138 (11h1m23.192139989s) [upload=11462]: Done uploading segments... Aug 7 20:41:01.448102791 (11h1m23.192159644s) [upload=11462]: Upload finished. err: <nil> Aug 7 20:41:25.522931110 (11h1m47.266987978s) [upload=11468]: Starting upload Aug 7 20:41:25.523039051 (11h1m47.267095924s) [upload=11468]: Done starting upload Aug 7 20:41:25.523181759 (11h1m47.267238621s) [upload=11468]: Uploading segments... Aug 7 20:41:25.523403073 (11h1m47.267459944s) [upload=11468]: Got next segment. Inline: true Aug 7 20:41:25.523413059 (11h1m47.26746993s) [upload=11468]: Waiting for error group managing segments... Aug 7 20:41:25.523492519 (11h1m47.267549389s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:25.523498700 (11h1m47.267555566s) [upload=11468]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:25.523502761 (11h1m47.267559629s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:25.840530122 (11h1m47.584586983s) [upload=11468]: Done uploading segments... Aug 7 20:41:25.840544406 (11h1m47.584601253s) [upload=11468]: Upload finished. err: <nil> Aug 7 20:41:49.037046771 (11h2m10.781103621s) [upload=11474]: Starting upload Aug 7 20:41:49.037108376 (11h2m10.781165228s) [upload=11474]: Done starting upload Aug 7 20:41:49.037181045 (11h2m10.781237895s) [upload=11474]: Uploading segments... Aug 7 20:41:49.037238734 (11h2m10.781295606s) [upload=11474]: Got next segment. Inline: true Aug 7 20:41:49.037244163 (11h2m10.781301028s) [upload=11474]: Waiting for error group managing segments... Aug 7 20:41:49.037309765 (11h2m10.781366628s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:49.037314702 (11h2m10.781371562s) [upload=11474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:49.037317447 (11h2m10.781374307s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:49.205260584 (11h2m10.949317434s) [upload=11474]: Done uploading segments... Aug 7 20:41:49.205278299 (11h2m10.94933515s) [upload=11474]: Upload finished. err: <nil> Duration: 857.493631ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 30.000537393s FAIL (2025-08-07 21:23:36 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000566978s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [...........................................X..XXXXXXXXXXXXXX] duration: 30.000630936s FAIL (2025-08-07 21:23:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000646168s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 6m49.512231112s FAIL (2025-08-07 21:23:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== g segments... Aug 7 21:16:45.007716349 (11h37m6.7517732s) [upload=12056]: Got next segment. Inline: false Aug 7 21:16:45.007801447 (11h37m6.751858349s) [upload=12056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:45.007808670 (11h37m6.751865533s) [upload=12056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:17:31.722627244 (11h37m53.466684104s) [seg_pos={0 0}, upload=12056]: Begin upload segment... Aug 7 21:17:31.722687690 (11h37m53.46674456s) [seg_pos={0 0}, upload=12056]: Done begin upload segment. Aug 7 21:17:31.722712882 (11h37m53.466769748s) [noise=true , address=104.36.82.38:4700 , piece_id=RKXSWNPNDE77WSUTVE5ORSPLF2CP73FWMTKYNH2WETS7KVRPDYRQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.722726328 (11h37m53.466783196s) [noise=true , address=45.91.116.9:4286 , piece_id=K6JVV2LJR3624665CRKJRFPFK3W2SUTL3SHPJPIMX42Y5QSK4NWA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.722783177 (11h37m53.466840067s) [noise=true , address=104.36.82.18:4385 , piece_id=P55LU5RIEFOXVJMT5ECSZVBJVR3EKTZMTQUYYZADAMWLXQDD4PVA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.723152014 (11h37m53.467208877s) [noise=true , address=79.127.201.215:4354 , piece_id=ZSCV73YKDLJGMUTZC25BKEDN42U4MTK4YXSTQLAL7CJUHGKPC6ZQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.723164484 (11h37m53.467221343s) [noise=true , address=104.36.82.11:4117 , piece_id=WYFZCITVSFFKJWASQ3NJATWLGMNENL3HHJI7UDFARGQKV4NJEOPQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.723613232 (11h37m53.467670101s) [noise=true , address=104.36.82.58:4283 , piece_id=BGC2FWKKRJACRMUB3W5ZIHLCSHV2GQMIMJQAVLYHTY7O3TLEEOXQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.723920353 (11h37m53.467977215s) [noise=true , address=104.36.82.35:4596 , piece_id=T7IXOFRCQBNKM4KYP26RYXCVJ23BTNQVAWAOS6RJ5FU5PXQNH2VA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.724224848 (11h37m53.468281719s) [noise=true , address=45.91.116.6:4983 , piece_id=SQG4PIM6NRCBNWGNDEKGEAT5NKC5ZPA2AIYMBNVECK2PIXDDK6VA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.724565477 (11h37m53.468622346s) [noise=true , address=104.36.82.52:4114 , piece_id=DOKPKLWNIR5U3DL4H57J2TWFQPCQPS5JYSHXHATQVNCM4MDM3UIQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.724862177 (11h37m53.468919042s) [noise=true , address=104.36.82.62:4410 , piece_id=J2TWQYM2SQC5CYHQTLXDWRX7PF5C2T6ANYJ3NGE3KHZJBJZGR4AQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.725090136 (11h37m53.469146998s) [noise=true , address=104.36.82.61:4386 , piece_id=DHHIO7VLBG4O4J4KZ33DLN7L557WGMKNIOP7M4HTOGKTR5UZWSKA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.725329528 (11h37m53.469386388s) [noise=true , address=45.91.116.5:4307 , piece_id=5U7LO24VJLOCITSG5WNVO7H5XHUP4W254TNSPII6DIBI2BQPMBQA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.725607912 (11h37m53.469664775s) [noise=true , address=104.36.82.36:4618 , piece_id=IDWBF76EB4KMI2U636Q3NJZTY3TLBHWLFC5LAKER6LET7VEUQEDQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.725840955 (11h37m53.469897818s) [noise=true , address=104.36.82.9:4018 , piece_id=OQ4DZQI5SLAWF64WJJD2SWA4NZJG63JCKK6VRW6ZWR2QTX27UKCA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.726098568 (11h37m53.470155431s) [noise=true , address=104.36.82.37:4665 , piece_id=C45VO2IWK4LW4J23K4BJT6KQT3DMXKC5NZ4H4H7TD3V4WEKCUASQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.726341913 (11h37m53.470398777s) [noise=true , address=104.36.82.59:4301 , piece_id=OE2HM5UBT5G7SAELRRZVDHMNSVTC3SF6PIOOOELWXFRYGD4TESYA, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.726645507 (11h37m53.470702371s) [noise=true , address=79.127.219.47:4058 , piece_id=7OIUVNDUPCYRK5CCQBI6KOBMYRKCKHPRGIVC5PYKDDW3ZIUOBIFQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:31.726938117 (11h37m53.470994979s) [noise=true , address=45.91.116.14:4200 , piece_id=LTYXWK7RBFXWUCQOSQPQC3YZSWD55CXGL76SNSQRJCBO7HEJLRXQ, seg_pos={0 0}, upload=12056]: Uploading piece... Aug 7 21:17:32.333624838 (11h37m54.077681691s) [upload=12056]: Next returned nil segment Aug 7 21:17:32.333642519 (11h37m54.07769937s) [upload=12056]: Waiting for error group managing segments... Aug 7 21:17:32.499764814 (11h37m54.243821675s) [noise=true , address=104.36.82.35:4596 , piece_id=T7IXOFRCQBNKM4KYP26RYXCVJ23BTNQVAWAOS6RJ5FU5PXQNH2VA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.500372993 (11h37m54.244429854s) [noise=true , address=79.127.219.47:4058 , piece_id=7OIUVNDUPCYRK5CCQBI6KOBMYRKCKHPRGIVC5PYKDDW3ZIUOBIFQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.527953393 (11h37m54.272010288s) [noise=true , address=104.36.82.38:4700 , piece_id=RKXSWNPNDE77WSUTVE5ORSPLF2CP73FWMTKYNH2WETS7KVRPDYRQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.528587479 (11h37m54.272644333s) [noise=true , address=104.36.82.36:4618 , piece_id=IDWBF76EB4KMI2U636Q3NJZTY3TLBHWLFC5LAKER6LET7VEUQEDQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.532068888 (11h37m54.276125739s) [noise=true , address=104.36.82.58:4283 , piece_id=BGC2FWKKRJACRMUB3W5ZIHLCSHV2GQMIMJQAVLYHTY7O3TLEEOXQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.535806115 (11h37m54.279862977s) [noise=true , address=104.36.82.52:4114 , piece_id=DOKPKLWNIR5U3DL4H57J2TWFQPCQPS5JYSHXHATQVNCM4MDM3UIQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.549937686 (11h37m54.293994536s) [noise=true , address=104.36.82.11:4117 , piece_id=WYFZCITVSFFKJWASQ3NJATWLGMNENL3HHJI7UDFARGQKV4NJEOPQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.578063947 (11h37m54.322120801s) [noise=true , address=45.91.116.14:4200 , piece_id=LTYXWK7RBFXWUCQOSQPQC3YZSWD55CXGL76SNSQRJCBO7HEJLRXQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.584491919 (11h37m54.328548767s) [noise=true , address=104.36.82.61:4386 , piece_id=DHHIO7VLBG4O4J4KZ33DLN7L557WGMKNIOP7M4HTOGKTR5UZWSKA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.589474263 (11h37m54.333531109s) [noise=true , address=104.36.82.62:4410 , piece_id=J2TWQYM2SQC5CYHQTLXDWRX7PF5C2T6ANYJ3NGE3KHZJBJZGR4AQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.599981486 (11h37m54.344038336s) [noise=true , address=104.36.82.59:4301 , piece_id=OE2HM5UBT5G7SAELRRZVDHMNSVTC3SF6PIOOOELWXFRYGD4TESYA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.627542635 (11h37m54.371599485s) [noise=true , address=45.91.116.9:4286 , piece_id=K6JVV2LJR3624665CRKJRFPFK3W2SUTL3SHPJPIMX42Y5QSK4NWA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.695607895 (11h37m54.439664748s) [noise=true , address=45.91.116.5:4307 , piece_id=5U7LO24VJLOCITSG5WNVO7H5XHUP4W254TNSPII6DIBI2BQPMBQA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.697853180 (11h37m54.44191004s) [noise=true , address=104.36.82.37:4665 , piece_id=C45VO2IWK4LW4J23K4BJT6KQT3DMXKC5NZ4H4H7TD3V4WEKCUASQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: <nil> Aug 7 21:17:32.697876596 (11h37m54.441933447s) [seg_pos={0 0}, upload=12056]: Segment reached optimal threshold of 14 pieces. Aug 7 21:17:32.698050022 (11h37m54.442106873s) [noise=true , address=45.91.116.6:4983 , piece_id=SQG4PIM6NRCBNWGNDEKGEAT5NKC5ZPA2AIYMBNVECK2PIXDDK6VA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122trJbDHuqeBMQHZT97HP2PgkeuLyUiDiuRm4nayu6ZTH7tat6): protocol: expected piece hash; context canceled; EOF Aug 7 21:17:32.698076888 (11h37m54.442133747s) [noise=true , address=79.127.201.215:4354 , piece_id=ZSCV73YKDLJGMUTZC25BKEDN42U4MTK4YXSTQLAL7CJUHGKPC6ZQ, seg_pos={0 0}, upload=12056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13Jtv8kL5R9oVro2D4ioqsznoJBXyNgJLdj1FcrGN3WWH45rkP): protocol: expected piece hash; context canceled; EOF Aug 7 21:17:32.698154719 (11h37m54.442211568s) [noise=true , address=104.36.82.18:4385 , piece_id=P55LU5RIEFOXVJMT5ECSZVBJVR3EKTZMTQUYYZADAMWLXQDD4PVA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123PZh1ieWYoc9U5CRrRzV1ZTpGYmjLmh5CpRy1NRC8wiLkumbs): protocol: expected piece hash; context canceled; EOF Aug 7 21:17:32.698262049 (11h37m54.442318898s) [noise=true , address=104.36.82.9:4018 , piece_id=OQ4DZQI5SLAWF64WJJD2SWA4NZJG63JCKK6VRW6ZWR2QTX27UKCA, seg_pos={0 0}, upload=12056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFRHbXjTaxoEKc5gbQWYt6jwUg8jknScBZpc8PdKNSZhYqUqP): protocol: write tcp 139.59.144.22:47050->104.36.82.9:4018: use of closed network connection; write tcp 139.59.144.22:47050->104.36.82.9:4018: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47050->104.36.82.9:4018: use of closed network connection Aug 7 21:17:32.698294367 (11h37m54.442351219s) [seg_pos={0 0}, upload=12056]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:17:32.698397017 (11h37m54.442453885s) [upload=12056]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:17:32.698404035 (11h37m54.442460903s) [upload=12056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:34.247964577 (11h43m55.99202143s) [upload=12056]: Done uploading segments... Aug 7 21:23:34.247989370 (11h43m55.992046231s) [upload=12056]: Upload finished. err: metaclient: context deadline exceeded Duration: 6m49.512248041s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................X......................X..........] duration: 2.681411815s FAIL (2025-08-07 21:23:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== oad finished. err: <nil> Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments... Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment... Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment. Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments... Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces. Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments... Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil> Duration: 2.681448636s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 109.73491ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:23:16 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 109.763541ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [...................................................X.XXXXXXX] duration: 1m0.000612849s FAIL (2025-08-07 21:23:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 1992]: Done starting upload Aug 7 21:12:47.792248444 (11h33m9.536305307s) [upload=11992]: Uploading segments... Aug 7 21:12:47.792351067 (11h33m9.536407934s) [upload=11992]: Got next segment. Inline: true Aug 7 21:12:47.792359617 (11h33m9.536416486s) [upload=11992]: Waiting for error group managing segments... Aug 7 21:12:47.792414618 (11h33m9.536471489s) [upload=11992]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:47.792420367 (11h33m9.536477238s) [upload=11992]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:47.792423893 (11h33m9.536480763s) [upload=11992]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:12:47.975624008 (11h33m9.71968088s) [upload=11992]: Done uploading segments... Aug 7 21:12:47.975648860 (11h33m9.71970573s) [upload=11992]: Upload finished. err: <nil> Aug 7 21:14:01.226859239 (11h34m22.970916111s) [upload=12013]: Starting upload Aug 7 21:14:01.227003646 (11h34m22.971060518s) [upload=12013]: Done starting upload Aug 7 21:14:01.227141194 (11h34m22.971198047s) [upload=12013]: Uploading segments... Aug 7 21:14:01.227219219 (11h34m22.971276084s) [upload=12013]: Got next segment. Inline: true Aug 7 21:14:01.227227170 (11h34m22.971284041s) [upload=12013]: Waiting for error group managing segments... Aug 7 21:14:01.227295298 (11h34m22.971352162s) [upload=12013]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:01.227301580 (11h34m22.971358443s) [upload=12013]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:01.227305300 (11h34m22.971362163s) [upload=12013]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:00.953618154 (11h35m22.697675004s) [upload=12013]: Done uploading segments... Aug 7 21:15:00.953640566 (11h35m22.697697426s) [upload=12013]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:01.394116473 (11h35m23.138173339s) [upload=12030]: Starting upload Aug 7 21:15:01.394242598 (11h35m23.138299459s) [upload=12030]: Done starting upload Aug 7 21:15:01.394373467 (11h35m23.138430318s) [upload=12030]: Uploading segments... Aug 7 21:15:01.394459055 (11h35m23.138515928s) [upload=12030]: Got next segment. Inline: true Aug 7 21:15:01.394467128 (11h35m23.138523996s) [upload=12030]: Waiting for error group managing segments... Aug 7 21:15:01.394548041 (11h35m23.138604915s) [upload=12030]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:01.394554816 (11h35m23.13861169s) [upload=12030]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:01.394559518 (11h35m23.138616392s) [upload=12030]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:03.584668408 (11h35m25.328725286s) [upload=12030]: Done uploading segments... Aug 7 21:15:03.584713689 (11h35m25.32877056s) [upload=12030]: Upload finished. err: <nil> Aug 7 21:15:54.636790163 (11h36m16.380847013s) [upload=12044]: Starting upload Aug 7 21:15:54.636877707 (11h36m16.380934638s) [upload=12044]: Done starting upload Aug 7 21:15:54.636936350 (11h36m16.3809932s) [upload=12044]: Uploading segments... Aug 7 21:15:54.636993824 (11h36m16.381050685s) [upload=12044]: Got next segment. Inline: true Aug 7 21:15:54.636999384 (11h36m16.381056244s) [upload=12044]: Waiting for error group managing segments... Aug 7 21:15:54.637044877 (11h36m16.381101727s) [upload=12044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:54.637050662 (11h36m16.381107522s) [upload=12044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:54.637053905 (11h36m16.381110766s) [upload=12044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:54.359175610 (11h37m16.103232476s) [upload=12044]: Done uploading segments... Aug 7 21:16:54.359203156 (11h37m16.103260018s) [upload=12044]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:54.796260076 (11h37m16.54031694s) [upload=12060]: Starting upload Aug 7 21:16:54.796405798 (11h37m16.540462658s) [upload=12060]: Done starting upload Aug 7 21:16:54.796514557 (11h37m16.540571407s) [upload=12060]: Uploading segments... Aug 7 21:16:54.796581012 (11h37m16.540637875s) [upload=12060]: Got next segment. Inline: true Aug 7 21:16:54.796586298 (11h37m16.540643158s) [upload=12060]: Waiting for error group managing segments... Aug 7 21:16:54.796657533 (11h37m16.540714383s) [upload=12060]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:54.796663030 (11h37m16.540719894s) [upload=12060]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:54.796669424 (11h37m16.540726289s) [upload=12060]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:17:54.519327140 (11h38m16.263384004s) [upload=12060]: Done uploading segments... Aug 7 21:17:54.519368022 (11h38m16.263424882s) [upload=12060]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:54.944498267 (11h38m16.688555136s) [upload=12074]: Starting upload Aug 7 21:17:54.944617406 (11h38m16.688674267s) [upload=12074]: Done starting upload Aug 7 21:17:54.944695430 (11h38m16.688752285s) [upload=12074]: Uploading segments... Aug 7 21:17:54.944771865 (11h38m16.688828736s) [upload=12074]: Got next segment. Inline: true Aug 7 21:17:54.944781252 (11h38m16.688838123s) [upload=12074]: Waiting for error group managing segments... Aug 7 21:17:54.944841042 (11h38m16.688897916s) [upload=12074]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:54.944848212 (11h38m16.688905079s) [upload=12074]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:54.944851963 (11h38m16.688908827s) [upload=12074]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:54.673864467 (11h39m16.417921315s) [upload=12074]: Done uploading segments... Aug 7 21:18:54.673885859 (11h39m16.417942708s) [upload=12074]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:55.119912759 (11h39m16.863969619s) [upload=12087]: Starting upload Aug 7 21:18:55.120013170 (11h39m16.864070018s) [upload=12087]: Done starting upload Aug 7 21:18:55.120094735 (11h39m16.864151596s) [upload=12087]: Uploading segments... Aug 7 21:18:55.120142836 (11h39m16.864199705s) [upload=12087]: Got next segment. Inline: true Aug 7 21:18:55.120147776 (11h39m16.864204637s) [upload=12087]: Waiting for error group managing segments... Aug 7 21:18:55.120201712 (11h39m16.864258573s) [upload=12087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:55.120207674 (11h39m16.864264534s) [upload=12087]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:55.120211321 (11h39m16.864268183s) [upload=12087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:54.844368358 (11h40m16.588425206s) [upload=12087]: Done uploading segments... Aug 7 21:19:54.844395070 (11h40m16.588451926s) [upload=12087]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:04.793025148 (11h40m26.537081997s) [upload=12103]: Starting upload Aug 7 21:20:04.793120317 (11h40m26.53717719s) [upload=12103]: Done starting upload Aug 7 21:20:04.793244612 (11h40m26.537301459s) [upload=12103]: Uploading segments... Aug 7 21:20:04.793331764 (11h40m26.537388638s) [upload=12103]: Got next segment. Inline: true Aug 7 21:20:04.793339748 (11h40m26.537396613s) [upload=12103]: Waiting for error group managing segments... Aug 7 21:20:04.793417981 (11h40m26.537474831s) [upload=12103]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:04.793424412 (11h40m26.537481262s) [upload=12103]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:04.793427866 (11h40m26.537484716s) [upload=12103]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:04.512178875 (11h41m26.256235737s) [upload=12103]: Done uploading segments... Aug 7 21:21:04.512205338 (11h41m26.256262189s) [upload=12103]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:14.239199000 (11h41m35.983255849s) [upload=12119]: Starting upload Aug 7 21:21:14.239282320 (11h41m35.983339189s) [upload=12119]: Done starting upload Aug 7 21:21:14.239356308 (11h41m35.98341316s) [upload=12119]: Uploading segments... Aug 7 21:21:14.239528709 (11h41m35.983585615s) [upload=12119]: Got next segment. Inline: true Aug 7 21:21:14.239538735 (11h41m35.983595603s) [upload=12119]: Waiting for error group managing segments... Aug 7 21:21:14.239621427 (11h41m35.983678288s) [upload=12119]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:14.239629030 (11h41m35.983685885s) [upload=12119]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:14.239632488 (11h41m35.98368934s) [upload=12119]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:13.962302553 (11h42m35.706359403s) [upload=12119]: Done uploading segments... Aug 7 21:22:13.962327972 (11h42m35.706384822s) [upload=12119]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:14.360206935 (11h42m36.104263785s) [upload=12134]: Starting upload Aug 7 21:22:14.360271125 (11h42m36.104327976s) [upload=12134]: Done starting upload Aug 7 21:22:14.360336747 (11h42m36.104393607s) [upload=12134]: Uploading segments... Aug 7 21:22:14.360380199 (11h42m36.104437068s) [upload=12134]: Got next segment. Inline: true Aug 7 21:22:14.360385659 (11h42m36.10444252s) [upload=12134]: Waiting for error group managing segments... Aug 7 21:22:14.360431265 (11h42m36.104488116s) [upload=12134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:14.360437033 (11h42m36.104493884s) [upload=12134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:14.360440534 (11h42m36.104497395s) [upload=12134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:14.088328715 (11h43m35.832385565s) [upload=12134]: Done uploading segments... Aug 7 21:23:14.088350462 (11h43m35.832407323s) [upload=12134]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.000628912s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 30.000630825s FAIL (2025-08-07 21:23:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000658633s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................X......................X.........] duration: 3.144504057s FAIL (2025-08-07 21:23:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== oad finished. err: <nil> Aug 7 21:14:53.365419244 (11h35m15.109476095s) [upload=12029]: Starting upload Aug 7 21:14:53.365483423 (11h35m15.109540274s) [upload=12029]: Done starting upload Aug 7 21:14:53.365847548 (11h35m15.109904403s) [upload=12029]: Uploading segments... Aug 7 21:14:53.365943593 (11h35m15.110000458s) [upload=12029]: Got next segment. Inline: false Aug 7 21:14:53.366001458 (11h35m15.110058308s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:53.366007419 (11h35m15.110064269s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:53.565386761 (11h35m15.309443632s) [seg_pos={0 0}, upload=12029]: Begin upload segment... Aug 7 21:14:53.565453585 (11h35m15.309510436s) [seg_pos={0 0}, upload=12029]: Done begin upload segment. Aug 7 21:14:53.565481091 (11h35m15.309537957s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.565521443 (11h35m15.309578305s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566289759 (11h35m15.310346619s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566595379 (11h35m15.31065223s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566764972 (11h35m15.310821839s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.566955689 (11h35m15.31101255s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567166790 (11h35m15.311223672s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567257141 (11h35m15.311314011s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567695963 (11h35m15.311752829s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.567729634 (11h35m15.311786511s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568042033 (11h35m15.312098893s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568093826 (11h35m15.312150693s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568392535 (11h35m15.312449404s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568543778 (11h35m15.312600643s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568718488 (11h35m15.312775348s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.568895867 (11h35m15.31295273s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.569229978 (11h35m15.313286841s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:53.570109875 (11h35m15.314166736s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Uploading piece... Aug 7 21:14:54.148674771 (11h35m15.892731622s) [upload=12029]: Next returned nil segment Aug 7 21:14:54.148691178 (11h35m15.892748028s) [upload=12029]: Waiting for error group managing segments... Aug 7 21:14:54.307599195 (11h35m16.051656048s) [noise=true , address=104.36.82.23:4468 , piece_id=FUQL5RAGCLECSKKYSNEP3ZNUIEKCCHG2CLWBHWTKL6O2QT23UF7A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.317245558 (11h35m16.061302413s) [noise=true , address=104.36.82.41:4793 , piece_id=UQRKUBTVEX357CCO2LG3J6HAEZYTD4AH4SJYDMPI7JV5VEHWKVLQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.327537824 (11h35m16.071594686s) [noise=true , address=104.36.82.60:4358 , piece_id=3EAVHMKYL2CLWDZX3DFAGX7ZRPFEKG5IA5IOVXS4QK347T5VTOLA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.335711829 (11h35m16.079768688s) [noise=true , address=104.36.82.53:4143 , piece_id=QHDKVDXS56LDT53FKQQ57KUFIY4TYO6W3OGOQYNBKLFZ6J7URK3Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.340624043 (11h35m16.084680896s) [noise=true , address=104.36.82.56:4229 , piece_id=J3TJ6C45D647QCYH2LP4NEZDFCW22JUPNZAFGVVEW26FMXDEUBQA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.347786380 (11h35m16.091843234s) [noise=true , address=104.36.82.8:4975 , piece_id=OT2R76IEADI5LFR7SDCUTYD3YIBZHFZG6TQTWFUK6JOSBS4MXU2A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.360886940 (11h35m16.104943808s) [noise=true , address=45.91.116.6:4002 , piece_id=UTJDKYUZI3F3KD5ZNSSKMYIZPZKZFPZDDJYNENEFIPIOZ3IX6Q3A, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.382665858 (11h35m16.126722724s) [noise=true , address=104.36.82.37:4686 , piece_id=EU5QDHW44OCPNEI3355XHD5VEVUCEFTO2LEDTH626KWZFMVIF3WA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.396988326 (11h35m16.141045179s) [noise=true , address=45.91.116.4:4944 , piece_id=H5M5J52YPWDBF27RKRY6TZE4DRFOTKBFQKIIJJFNJCYF4QZ2WO2Q, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.468138008 (11h35m16.212194859s) [noise=true , address=104.36.82.15:4263 , piece_id=MHQ54RTCV6BFUT6STUGEKOJNM2N3OQ7EXUE7XC44HP2IORHTABEA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.474865865 (11h35m16.218922761s) [noise=true , address=104.36.82.5:4859 , piece_id=KNIFV22KO2KVKCYTXUKCSZ774GMKDEYLY2AZQBUOFCGTVE2GNGKQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.664449958 (11h35m16.408506808s) [noise=true , address=104.36.82.59:4311 , piece_id=YW3HCFGQ7V5GSOEASEJ75A7RNT5URMVEW7RNPIHKYTJOPTXLRCZQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.806820897 (11h35m16.550877751s) [noise=true , address=193.149.164.143:4699 , piece_id=3QIF7NXRJXYHITN3GQ6DEAAVQ5FNL3F7PPUXKJBR72BHIUSGHUVQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826156011 (11h35m16.570212865s) [noise=true , address=64.49.13.140:4510 , piece_id=AOLVILWMK2UVHN4ZHXK4B4XKJRSZSYXPZL4YN65TWUYQEONI55MQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: <nil> Aug 7 21:14:54.826199084 (11h35m16.570255944s) [seg_pos={0 0}, upload=12029]: Segment reached optimal threshold of 14 pieces. Aug 7 21:14:54.826382478 (11h35m16.570439338s) [noise=true , address=79.127.205.234:4602 , piece_id=MX35WEO2ET72CJP2BKTK7WEPQBARCOE27M7P3H7OZXPO4KCWHJKA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XeaD9pExNha3FaRaCyyfTHBeQMrmP19MHR135yT8a7Uo4eUyZ): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826407613 (11h35m16.570464464s) [noise=true , address=64.49.13.26:4956 , piece_id=WZU2GAEHVVGHYAKGI6AZ27VAUOTQ2KDAQAU2X4DAVNX2BIVZETRA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TYHhjpfyBKkenSwviyQCUZpSBbnaFCGBFRkM4fyKLDuxKx4Ju): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826504181 (11h35m16.570561027s) [noise=true , address=79.127.205.233:4646 , piece_id=KXPQQXRFFQBH7DQKOKURS4EN47YIOZWAM3N7PCIA4CYA4FVJ64SQ, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fnBRPbkdAGVB8exwLJBBtpf5bpAN8Dv86mgGf3YVMEUfX4Hdc): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826519435 (11h35m16.570576285s) [noise=true , address=104.36.82.61:4380 , piece_id=DDLKPWKAAGKDTYFNRTPUIGL7ZNTGJRQJI4EXICX3Q4KTUSR3HYYA, seg_pos={0 0}, upload=12029]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MqrQPmbGWnPhLYSQKxX1eKASgv9SdUM2p4WXfQYBcENHh9VPG): protocol: expected piece hash; context canceled; EOF Aug 7 21:14:54.826544440 (11h35m16.57060129s) [seg_pos={0 0}, upload=12029]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:14:54.826652867 (11h35m16.570709735s) [upload=12029]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:14:54.826659817 (11h35m16.570716781s) [upload=12029]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:54.989374282 (11h35m16.733431131s) [upload=12029]: Done uploading segments... Aug 7 21:14:54.989392341 (11h35m16.73344919s) [upload=12029]: Upload finished. err: <nil> Duration: 3.144554355s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 30.00115814s FAIL (2025-08-07 21:23:06 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001210243s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [............................................X..XXXXXXXXXXXXX] duration: 30.001114895s FAIL (2025-08-07 21:23:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001141838s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [...................................................XX.......] duration: 352.032396ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:23:00 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 352.068456ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 1m0.001623706s FAIL (2025-08-07 21:22:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001644391s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 296.382006ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:22:56 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 296.459727ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 30.000633993s FAIL (2025-08-07 21:22:52 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000666169s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X.......X..................................................] duration: 853.246254ms FAIL (2025-08-07 21:22:48 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== 68895 (10h54m20.003725754s) [upload=11347]: Done uploading segments... Aug 7 20:33:58.259694049 (10h54m20.00375091s) [upload=11347]: Upload finished. err: <nil> Aug 7 20:34:42.291300370 (10h55m4.035357231s) [upload=11357]: Starting upload Aug 7 20:34:42.291392440 (10h55m4.035449315s) [upload=11357]: Done starting upload Aug 7 20:34:42.291505939 (10h55m4.035562788s) [upload=11357]: Uploading segments... Aug 7 20:34:42.291587798 (10h55m4.035644667s) [upload=11357]: Got next segment. Inline: true Aug 7 20:34:42.291610428 (10h55m4.035667289s) [upload=11357]: Waiting for error group managing segments... Aug 7 20:34:42.291660078 (10h55m4.035716927s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:34:42.291665407 (10h55m4.035722257s) [upload=11357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:34:42.291668818 (10h55m4.035725679s) [upload=11357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:34:42.445169337 (10h55m4.189226185s) [upload=11357]: Done uploading segments... Aug 7 20:34:42.445187662 (10h55m4.189244508s) [upload=11357]: Upload finished. err: <nil> Aug 7 20:35:56.993979713 (10h56m18.738036563s) [upload=11378]: Starting upload Aug 7 20:35:56.994042734 (10h56m18.738099594s) [upload=11378]: Done starting upload Aug 7 20:35:56.994133564 (10h56m18.738190418s) [upload=11378]: Uploading segments... Aug 7 20:35:56.994188836 (10h56m18.738245705s) [upload=11378]: Got next segment. Inline: true Aug 7 20:35:56.994194253 (10h56m18.738251113s) [upload=11378]: Waiting for error group managing segments... Aug 7 20:35:56.994255059 (10h56m18.738311921s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:35:56.994261721 (10h56m18.738318586s) [upload=11378]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:35:56.994266228 (10h56m18.73832309s) [upload=11378]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:35:57.196915243 (10h56m18.940972094s) [upload=11378]: Done uploading segments... Aug 7 20:35:57.196933580 (10h56m18.940990431s) [upload=11378]: Upload finished. err: <nil> Aug 7 20:36:48.383926585 (10h57m10.127983431s) [upload=11392]: Starting upload Aug 7 20:36:48.383977279 (10h57m10.128034139s) [upload=11392]: Done starting upload Aug 7 20:36:48.384080928 (10h57m10.128137781s) [upload=11392]: Uploading segments... Aug 7 20:36:48.384124913 (10h57m10.128181781s) [upload=11392]: Got next segment. Inline: true Aug 7 20:36:48.384130188 (10h57m10.128187037s) [upload=11392]: Waiting for error group managing segments... Aug 7 20:36:48.384183298 (10h57m10.128240148s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:36:48.384188890 (10h57m10.128245741s) [upload=11392]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:36:48.384192353 (10h57m10.128249203s) [upload=11392]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:36:48.597716681 (10h57m10.341773532s) [upload=11392]: Done uploading segments... Aug 7 20:36:48.597735626 (10h57m10.341792476s) [upload=11392]: Upload finished. err: <nil> Aug 7 20:37:59.886756865 (10h58m21.630813711s) [upload=11411]: Starting upload Aug 7 20:37:59.886810662 (10h58m21.630867522s) [upload=11411]: Done starting upload Aug 7 20:37:59.886870714 (10h58m21.630927561s) [upload=11411]: Uploading segments... Aug 7 20:37:59.886906763 (10h58m21.630963634s) [upload=11411]: Got next segment. Inline: true Aug 7 20:37:59.886911413 (10h58m21.630968259s) [upload=11411]: Waiting for error group managing segments... Aug 7 20:37:59.886948286 (10h58m21.631005132s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:37:59.886952923 (10h58m21.631009769s) [upload=11411]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:37:59.886957250 (10h58m21.631014096s) [upload=11411]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:38:00.073123067 (10h58m21.817179918s) [upload=11411]: Done uploading segments... Aug 7 20:38:00.073142248 (10h58m21.817199097s) [upload=11411]: Upload finished. err: <nil> Aug 7 20:39:06.192304563 (10h59m27.936361423s) [upload=11429]: Starting upload Aug 7 20:39:06.192379945 (10h59m27.936436804s) [upload=11429]: Done starting upload Aug 7 20:39:06.192450925 (10h59m27.936507772s) [upload=11429]: Uploading segments... Aug 7 20:39:06.192493866 (10h59m27.936550726s) [upload=11429]: Got next segment. Inline: true Aug 7 20:39:06.192499400 (10h59m27.93655626s) [upload=11429]: Waiting for error group managing segments... Aug 7 20:39:06.192549058 (10h59m27.936605918s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:06.192553658 (10h59m27.936610518s) [upload=11429]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:06.192556283 (10h59m27.936613143s) [upload=11429]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:06.410732058 (10h59m28.154788918s) [upload=11429]: Done uploading segments... Aug 7 20:39:06.410751072 (10h59m28.154807921s) [upload=11429]: Upload finished. err: <nil> Aug 7 20:39:38.189437550 (10h59m59.9334944s) [upload=11438]: Starting upload Aug 7 20:39:38.189494699 (10h59m59.933551565s) [upload=11438]: Done starting upload Aug 7 20:39:38.189570885 (10h59m59.933627735s) [upload=11438]: Uploading segments... Aug 7 20:39:38.189613828 (10h59m59.933670698s) [upload=11438]: Got next segment. Inline: true Aug 7 20:39:38.189618898 (10h59m59.933675758s) [upload=11438]: Waiting for error group managing segments... Aug 7 20:39:38.189671547 (10h59m59.933728396s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:39:38.189676923 (10h59m59.933733774s) [upload=11438]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:39:38.189680363 (10h59m59.933737214s) [upload=11438]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:39:38.339370287 (11h0m0.083427138s) [upload=11438]: Done uploading segments... Aug 7 20:39:38.339394965 (11h0m0.083451822s) [upload=11438]: Upload finished. err: <nil> Aug 7 20:40:13.049368895 (11h0m34.793425745s) [upload=11448]: Starting upload Aug 7 20:40:13.049412890 (11h0m34.793469743s) [upload=11448]: Done starting upload Aug 7 20:40:13.049490173 (11h0m34.793547027s) [upload=11448]: Uploading segments... Aug 7 20:40:13.049531553 (11h0m34.793588423s) [upload=11448]: Got next segment. Inline: true Aug 7 20:40:13.049536305 (11h0m34.79359316s) [upload=11448]: Waiting for error group managing segments... Aug 7 20:40:13.049606089 (11h0m34.793662937s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:40:13.049611863 (11h0m34.793668715s) [upload=11448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:40:13.049615487 (11h0m34.793672338s) [upload=11448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:40:13.251485869 (11h0m34.995542721s) [upload=11448]: Done uploading segments... Aug 7 20:40:13.251504252 (11h0m34.995561102s) [upload=11448]: Upload finished. err: <nil> Aug 7 20:41:01.300323709 (11h1m23.044380561s) [upload=11462]: Starting upload Aug 7 20:41:01.300386685 (11h1m23.044443554s) [upload=11462]: Done starting upload Aug 7 20:41:01.300471424 (11h1m23.044528277s) [upload=11462]: Uploading segments... Aug 7 20:41:01.300528867 (11h1m23.044585736s) [upload=11462]: Got next segment. Inline: true Aug 7 20:41:01.300534426 (11h1m23.044591286s) [upload=11462]: Waiting for error group managing segments... Aug 7 20:41:01.300592418 (11h1m23.044649268s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:01.300598189 (11h1m23.04465505s) [upload=11462]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:01.300601616 (11h1m23.044658476s) [upload=11462]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:01.448083138 (11h1m23.192139989s) [upload=11462]: Done uploading segments... Aug 7 20:41:01.448102791 (11h1m23.192159644s) [upload=11462]: Upload finished. err: <nil> Aug 7 20:41:25.522931110 (11h1m47.266987978s) [upload=11468]: Starting upload Aug 7 20:41:25.523039051 (11h1m47.267095924s) [upload=11468]: Done starting upload Aug 7 20:41:25.523181759 (11h1m47.267238621s) [upload=11468]: Uploading segments... Aug 7 20:41:25.523403073 (11h1m47.267459944s) [upload=11468]: Got next segment. Inline: true Aug 7 20:41:25.523413059 (11h1m47.26746993s) [upload=11468]: Waiting for error group managing segments... Aug 7 20:41:25.523492519 (11h1m47.267549389s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:25.523498700 (11h1m47.267555566s) [upload=11468]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:25.523502761 (11h1m47.267559629s) [upload=11468]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:25.840530122 (11h1m47.584586983s) [upload=11468]: Done uploading segments... Aug 7 20:41:25.840544406 (11h1m47.584601253s) [upload=11468]: Upload finished. err: <nil> Aug 7 20:41:49.037046771 (11h2m10.781103621s) [upload=11474]: Starting upload Aug 7 20:41:49.037108376 (11h2m10.781165228s) [upload=11474]: Done starting upload Aug 7 20:41:49.037181045 (11h2m10.781237895s) [upload=11474]: Uploading segments... Aug 7 20:41:49.037238734 (11h2m10.781295606s) [upload=11474]: Got next segment. Inline: true Aug 7 20:41:49.037244163 (11h2m10.781301028s) [upload=11474]: Waiting for error group managing segments... Aug 7 20:41:49.037309765 (11h2m10.781366628s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 20:41:49.037314702 (11h2m10.781371562s) [upload=11474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 20:41:49.037317447 (11h2m10.781374307s) [upload=11474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 20:41:49.205260584 (11h2m10.949317434s) [upload=11474]: Done uploading segments... Aug 7 20:41:49.205278299 (11h2m10.94933515s) [upload=11474]: Upload finished. err: <nil> Duration: 853.276882ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [....................................................XX......] duration: 320.608162ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:22:38 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 320.649755ms edge gateway vhost us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)