Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 428.43097ms FAIL (2025-07-14 23:50:32 UTC) orbiter warning: ======ERROR====== uplink: too many requests ======LOGS====== . Jul 14 23:47:30.334396460 (518h36m30.217810097s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.334541002 (518h36m30.217954628s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.334751321 (518h36m30.218164948s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.335060864 (518h36m30.218474493s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.335335126 (518h36m30.218748752s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.336922859 (518h36m30.220336495s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.340514312 (518h36m30.223927967s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.342052396 (518h36m30.225466052s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.485954867 (518h36m30.369368503s) [upload=414748]: Next returned nil segment Jul 14 23:47:30.485992786 (518h36m30.369406532s) [upload=414748]: Waiting for error group managing segments... Jul 14 23:47:30.494842357 (518h36m30.378255988s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.494930968 (518h36m30.378344595s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.496014314 (518h36m30.379427941s) [noise=true , address=89.149.201.147:4023 , piece_id=NQLJ6UGQTXWMNJ2XJSVQ2JPD2INWQNUPPQVHBYO6HPX6XX6HMWFA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.496680756 (518h36m30.380094372s) [noise=true , address=89.149.201.147:4006 , piece_id=JDGIM3TFHCOR7DT2EW4DHR4KKMYTBNJXEQQKI2YVB3OFIRXAMETA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.496801905 (518h36m30.380215532s) [noise=true , address=89.149.201.147:4019 , piece_id=3DMKAPD3M2A25PEKEWOOJCHV6CWKDUOZCZNPYDTQYBNSAML3MHCQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.497347115 (518h36m30.380760731s) [noise=true , address=89.149.201.147:4010 , piece_id=WCL5DLELLH7HWRF4NVXPG66W3PABG5LCQ7EI5QCHNGYL3WP26WSA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.497557301 (518h36m30.380970929s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498036118 (518h36m30.381449734s) [noise=true , address=89.149.201.147:4016 , piece_id=ZMUHQFGECGG6MKG7UWO4NWYOABOD6VYQ27JQNHSWGBCNY6RQOUCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498203330 (518h36m30.381616957s) [noise=true , address=185.167.204.205:4020 , piece_id=AF6L62IEU6KD2JIX7BL4VSIK6T7OCPETILSYGVGNS3MKX3XWTWKQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498564222 (518h36m30.381977836s) [noise=true , address=185.167.204.205:4023 , piece_id=B7OGRML6TDBWHQW7H5WZISNAEZLUTC4NUU6YJXLJG76ODSTTR2XA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498778308 (518h36m30.382191936s) [noise=true , address=81.171.14.70:4047 , piece_id=B2B7YV32YDVUG2L4Q6KV5FP63D6LIIVXEVWQEVQPLLO64NIWVK7A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.499060384 (518h36m30.382473998s) [noise=true , address=89.149.201.147:4017 , piece_id=OIUOJWKUHESMRGC2BHH45FNETDVKUMQTZHJICDYUDZCI66HS24EA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.499773462 (518h36m30.383187089s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.500485883 (518h36m30.383899502s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.500637044 (518h36m30.384050662s) [noise=true , address=89.149.201.147:4024 , piece_id=R6CF74YBNZ5NZHPCCBTUZ3PGHY56SAILYSNXV5L7FV37VORHXLZA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.500973891 (518h36m30.384387517s) [noise=true , address=81.171.14.70:4035 , piece_id=WMRQULXAT3UTOHK42S2ILFR7T7PICZUWCDRTMNOPCK2MKEZMI5LA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.501397526 (518h36m30.384811139s) [noise=true , address=89.149.201.147:4012 , piece_id=26IJ6NKKCUGV35XKHM4TISEX2CIWG3DNIHWYOHOMVMU4ANTCX6RA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.502337750 (518h36m30.385751362s) [noise=true , address=81.171.14.70:4030 , piece_id=FYBIQ4RINS5VAJS4VJKIJIREPWAZPESYAKSBKSZEX6OWXELKHJ3A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.502702474 (518h36m30.386116088s) [noise=true , address=81.171.14.70:4039 , piece_id=E4Q4U6JPWVZOWPDJI6A6AO7WQ72D2GOPMZQSYOHCWOOZVVE6NIKA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.503020550 (518h36m30.386434163s) [noise=true , address=81.171.14.70:4044 , piece_id=3XFKCWS4CN6C73YU54KJNACXZDBZUKBMSASVLAFGKOF5SGGB2UUA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.503356663 (518h36m30.386770277s) [noise=true , address=81.171.14.70:4031 , piece_id=GLUVAUI4ZVFE2C6OCNNRS3GVAFBJ6WG3ALYTNEBRSL6QBXA5B4LQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.504974620 (518h36m30.388388232s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.505290772 (518h36m30.388704384s) [noise=true , address=185.167.204.205:4027 , piece_id=4T7OLFAAULHVCZO7IOTX4CAJRA5J4ALHNB23B2SXPCSILECYTG6A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.505629943 (518h36m30.389043556s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.505933309 (518h36m30.389346921s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.507266987 (518h36m30.390680601s) [noise=true , address=89.149.201.147:4015 , piece_id=NPDGCLM2ZF64ZBOAPL2VT2XR4QMTFNZC3I43VVXLMGPCVER3F45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.507293406 (518h36m30.390707035s) [seg_pos={0 0}, upload=414748]: Segment reached optimal threshold of 26 pieces. Jul 14 23:47:30.507519964 (518h36m30.390933597s) [noise=true , address=81.171.14.70:4042 , piece_id=ASRUL4I2SLJEFFJLR6KBMGIAY3THJPR67YGCHBIMYNUFJKFRP2GA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD): protocol: expected piece hash; context canceled; EOF Jul 14 23:47:30.507609664 (518h36m30.391023293s) [noise=true , address=81.171.14.70:4029 , piece_id=LYKFSQWETSW6OJPHJYVD3LLP44S7RABUIBRHI5LMZSP2JJLDQIHQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to): protocol: expected piece hash; context canceled; EOF Jul 14 23:47:30.507634788 (518h36m30.391048419s) [seg_pos={0 0}, upload=414748]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Jul 14 23:47:30.507717732 (518h36m30.391131358s) [upload=414748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 14 23:47:30.507724547 (518h36m30.391138177s) [upload=414748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:47:30.547070340 (518h36m30.430483976s) [upload=414748]: Done uploading segments... Jul 14 23:47:30.547094478 (518h36m30.430508105s) [upload=414748]: Upload finished. err: <nil> Jul 14 23:47:30.863955222 (518h36m30.747368849s) [upload=414749]: Starting upload Jul 14 23:47:30.864023814 (518h36m30.747437456s) [upload=414749]: Done starting upload Jul 14 23:47:30.864126584 (518h36m30.747540202s) [upload=414749]: Uploading segments... Jul 14 23:47:30.864360135 (518h36m30.747773761s) [upload=414749]: Got next segment. Inline: false Jul 14 23:47:30.864426077 (518h36m30.747839704s) [upload=414749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:47:30.864431154 (518h36m30.747844779s) [upload=414749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 14 23:47:30.867084429 (518h36m30.750498058s) [upload=414749]: Next segment err: metaclient: Too Many Requests Jul 14 23:47:30.867113197 (518h36m30.750526824s) [upload=414749]: Done uploading segments... Jul 14 23:47:30.867126113 (518h36m30.750539725s) [upload=414749]: Upload finished. err: metaclient: Too Many Requests Duration: 428.456423ms core uplink eu-select-1 eu1-placement-15
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 537.910077ms FAIL (2025-07-14 23:50:28 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:54371->127.0.0.1:53: i/o timeout Duration: 537.966917ms edge authservice http bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.596234765s FAIL (2025-07-14 23:50:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.596294083s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 317.796081ms target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-14 23:50:21 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp: lookup uptime-checks-us-select.s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:36040->127.0.0.1:53: i/o timeout Duration: 317.820989ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.039011801s FAIL (2025-07-14 23:50:20 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/gateway-tests/from_do_prod_ops_orbiter_eu_0/1K?x-id=PutObject": dial tcp: lookup s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:57187->127.0.0.1:53: i/o timeout Duration: 1.039040378s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.242900491s FAIL (2025-07-14 23:50:15 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.2429229s core uplink us1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.492230317s FAIL (2025-07-14 23:50:15 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.492255456s core uplink ap1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 2.754760124s FAIL (2025-07-14 23:50:15 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:53438->127.0.0.1:53: i/o timeout Duration: 2.754785633s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 913.484696ms FAIL (2025-07-14 23:50:13 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load Jul 14 23:37:41.147633909 (518h26m41.031047542s) [upload=414628]: Done starting upload Jul 14 23:37:41.147700406 (518h26m41.031114026s) [upload=414628]: Uploading segments... Jul 14 23:37:41.147749349 (518h26m41.031162964s) [upload=414628]: Got next segment. Inline: true Jul 14 23:37:41.147757931 (518h26m41.031171548s) [upload=414628]: Waiting for error group managing segments... Jul 14 23:37:41.147803445 (518h26m41.031217064s) [upload=414628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:37:41.147810713 (518h26m41.031224329s) [upload=414628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:37:41.147814220 (518h26m41.031227838s) [upload=414628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:37:41.293868983 (518h26m41.177282609s) [upload=414628]: Done uploading segments... Jul 14 23:37:41.293899619 (518h26m41.177313246s) [upload=414628]: Upload finished. err: <nil> Jul 14 23:38:42.060547925 (518h27m41.943961552s) [upload=414641]: Starting upload Jul 14 23:38:42.060706705 (518h27m41.94412033s) [upload=414641]: Done starting upload Jul 14 23:38:42.060775681 (518h27m41.9441893s) [upload=414641]: Uploading segments... Jul 14 23:38:42.060850917 (518h27m41.944264562s) [upload=414641]: Got next segment. Inline: true Jul 14 23:38:42.060857684 (518h27m41.944271318s) [upload=414641]: Waiting for error group managing segments... Jul 14 23:38:42.060905522 (518h27m41.944319139s) [upload=414641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:38:42.060911671 (518h27m41.944325297s) [upload=414641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:38:42.060915517 (518h27m41.944329143s) [upload=414641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:38:42.214653360 (518h27m42.098066984s) [upload=414641]: Done uploading segments... Jul 14 23:38:42.214672288 (518h27m42.098085904s) [upload=414641]: Upload finished. err: <nil> Jul 14 23:39:37.897497561 (518h28m37.780911178s) [upload=414653]: Starting upload Jul 14 23:39:37.897653216 (518h28m37.781066839s) [upload=414653]: Done starting upload Jul 14 23:39:37.897770536 (518h28m37.781184151s) [upload=414653]: Uploading segments... Jul 14 23:39:37.897826735 (518h28m37.781240362s) [upload=414653]: Got next segment. Inline: true Jul 14 23:39:37.897832545 (518h28m37.78124616s) [upload=414653]: Waiting for error group managing segments... Jul 14 23:39:37.897874540 (518h28m37.781288156s) [upload=414653]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:39:37.897880498 (518h28m37.781294125s) [upload=414653]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:39:37.897884204 (518h28m37.78129783s) [upload=414653]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:39:38.174501392 (518h28m38.057915021s) [upload=414653]: Done uploading segments... Jul 14 23:39:38.174554097 (518h28m38.057967728s) [upload=414653]: Upload finished. err: <nil> Jul 14 23:40:47.713120393 (518h29m47.596534011s) [upload=414666]: Starting upload Jul 14 23:40:47.713181294 (518h29m47.596594929s) [upload=414666]: Done starting upload Jul 14 23:40:47.713333461 (518h29m47.596747074s) [upload=414666]: Uploading segments... Jul 14 23:40:47.713419846 (518h29m47.596833481s) [upload=414666]: Got next segment. Inline: true Jul 14 23:40:47.713427208 (518h29m47.596840876s) [upload=414666]: Waiting for error group managing segments... Jul 14 23:40:47.713492271 (518h29m47.596905915s) [upload=414666]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:40:47.713499814 (518h29m47.596913451s) [upload=414666]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:40:47.713503937 (518h29m47.596917586s) [upload=414666]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:40:47.868352520 (518h29m47.751766137s) [upload=414666]: Done uploading segments... Jul 14 23:40:47.868375465 (518h29m47.751789083s) [upload=414666]: Upload finished. err: <nil> Jul 14 23:41:00.797063324 (518h30m0.680476953s) [upload=414670]: Starting upload Jul 14 23:41:00.797141064 (518h30m0.68055469s) [upload=414670]: Done starting upload Jul 14 23:41:00.797212895 (518h30m0.680626511s) [upload=414670]: Uploading segments... Jul 14 23:41:00.797265690 (518h30m0.680679326s) [upload=414670]: Got next segment. Inline: true Jul 14 23:41:00.797270956 (518h30m0.680684572s) [upload=414670]: Waiting for error group managing segments... Jul 14 23:41:00.797328313 (518h30m0.680741939s) [upload=414670]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:41:00.797333853 (518h30m0.680747469s) [upload=414670]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:41:00.797337094 (518h30m0.68075072s) [upload=414670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:41:00.971853045 (518h30m0.855266664s) [upload=414670]: Done uploading segments... Jul 14 23:41:00.971870916 (518h30m0.855284535s) [upload=414670]: Upload finished. err: <nil> Jul 14 23:41:22.318635815 (518h30m22.202049441s) [upload=414675]: Starting upload Jul 14 23:41:22.318697752 (518h30m22.202111381s) [upload=414675]: Done starting upload Jul 14 23:41:22.318755348 (518h30m22.20216896s) [upload=414675]: Uploading segments... Jul 14 23:41:22.318804414 (518h30m22.202218058s) [upload=414675]: Got next segment. Inline: true Jul 14 23:41:22.318812150 (518h30m22.202225763s) [upload=414675]: Waiting for error group managing segments... Jul 14 23:41:22.318860496 (518h30m22.202274126s) [upload=414675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:41:22.318868059 (518h30m22.202281687s) [upload=414675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:41:22.318872383 (518h30m22.202286008s) [upload=414675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:41:22.468688511 (518h30m22.352102153s) [upload=414675]: Done uploading segments... Jul 14 23:41:22.468726173 (518h30m22.352139798s) [upload=414675]: Upload finished. err: <nil> Jul 14 23:42:39.877715214 (518h31m39.761128848s) [upload=414689]: Starting upload Jul 14 23:42:39.877790693 (518h31m39.76120433s) [upload=414689]: Done starting upload Jul 14 23:42:39.877900973 (518h31m39.761314587s) [upload=414689]: Uploading segments... Jul 14 23:42:39.878016572 (518h31m39.761430206s) [upload=414689]: Got next segment. Inline: true Jul 14 23:42:39.878023247 (518h31m39.761436881s) [upload=414689]: Waiting for error group managing segments... Jul 14 23:42:39.878073628 (518h31m39.761487264s) [upload=414689]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:42:39.878079532 (518h31m39.761493165s) [upload=414689]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:42:39.878083359 (518h31m39.761496973s) [upload=414689]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:42:40.027578980 (518h31m39.910992594s) [upload=414689]: Done uploading segments... Jul 14 23:42:40.027596526 (518h31m39.911010138s) [upload=414689]: Upload finished. err: <nil> Jul 14 23:42:40.961643522 (518h31m40.845057134s) [upload=414691]: Starting upload Jul 14 23:42:40.961719043 (518h31m40.845132672s) [upload=414691]: Done starting upload Jul 14 23:42:40.961792548 (518h31m40.84520616s) [upload=414691]: Uploading segments... Jul 14 23:42:40.961837433 (518h31m40.845251071s) [upload=414691]: Got next segment. Inline: true Jul 14 23:42:40.961851857 (518h31m40.845265486s) [upload=414691]: Waiting for error group managing segments... Jul 14 23:42:40.961890425 (518h31m40.845304037s) [upload=414691]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:42:40.961896055 (518h31m40.845309668s) [upload=414691]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:42:40.961898536 (518h31m40.845312147s) [upload=414691]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:42:41.122451811 (518h31m41.005865429s) [upload=414691]: Done uploading segments... Jul 14 23:42:41.122493314 (518h31m41.005906934s) [upload=414691]: Upload finished. err: <nil> Jul 14 23:43:34.049461829 (518h32m33.932875453s) [upload=414704]: Starting upload Jul 14 23:43:34.049588567 (518h32m33.933002206s) [upload=414704]: Done starting upload Jul 14 23:43:34.049735609 (518h32m33.933149224s) [upload=414704]: Uploading segments... Jul 14 23:43:34.049802945 (518h32m33.933216575s) [upload=414704]: Got next segment. Inline: true Jul 14 23:43:34.049810415 (518h32m33.933224046s) [upload=414704]: Waiting for error group managing segments... Jul 14 23:43:34.049877223 (518h32m33.93329084s) [upload=414704]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:43:34.049883389 (518h32m33.933297014s) [upload=414704]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:43:34.049887164 (518h32m33.933300789s) [upload=414704]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:43:34.197832086 (518h32m34.081245712s) [upload=414704]: Done uploading segments... Jul 14 23:43:34.197864776 (518h32m34.08127839s) [upload=414704]: Upload finished. err: <nil> Jul 14 23:43:56.290942463 (518h32m56.174356083s) [upload=414709]: Starting upload Jul 14 23:43:56.291049037 (518h32m56.174462655s) [upload=414709]: Done starting upload Jul 14 23:43:56.291143970 (518h32m56.174557592s) [upload=414709]: Uploading segments... Jul 14 23:43:56.291226174 (518h32m56.174639809s) [upload=414709]: Got next segment. Inline: true Jul 14 23:43:56.291231359 (518h32m56.174644972s) [upload=414709]: Waiting for error group managing segments... Jul 14 23:43:56.291289737 (518h32m56.17470335s) [upload=414709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:43:56.291295711 (518h32m56.174709323s) [upload=414709]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:43:56.291298840 (518h32m56.174712452s) [upload=414709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:43:56.428766698 (518h32m56.312180324s) [upload=414709]: Done uploading segments... Jul 14 23:43:56.428788768 (518h32m56.312202386s) [upload=414709]: Upload finished. err: <nil> Duration: 913.571057ms core uplink us-select-1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [...................X.................................X......] duration: 517.483228ms target: https://ap1.storj.io/login FAIL (2025-07-14 23:50:11 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp: lookup ap1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:60425->127.0.0.1:53: i/o timeout Duration: 517.505498ms core satellite ap1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 796.104429ms FAIL (2025-07-14 23:49:55 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:53438->127.0.0.1:53: i/o timeout Duration: 796.132808ms edge authservice http bgp ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.101806811s FAIL (2025-07-14 23:49:50 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/gateway-tests/from_do_prod_ops_orbiter_eu_0/1K?x-id=PutObject": dial tcp: lookup s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:57187->127.0.0.1:53: i/o timeout Duration: 1.101838223s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 389.727405ms FAIL (2025-07-14 23:49:43 UTC) orbiter warning: ======ERROR====== uplink: too many requests ======LOGS====== . Jul 14 23:47:30.334396460 (518h36m30.217810097s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.334541002 (518h36m30.217954628s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.334751321 (518h36m30.218164948s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.335060864 (518h36m30.218474493s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.335335126 (518h36m30.218748752s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.336922859 (518h36m30.220336495s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.340514312 (518h36m30.223927967s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.342052396 (518h36m30.225466052s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.485954867 (518h36m30.369368503s) [upload=414748]: Next returned nil segment Jul 14 23:47:30.485992786 (518h36m30.369406532s) [upload=414748]: Waiting for error group managing segments... Jul 14 23:47:30.494842357 (518h36m30.378255988s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.494930968 (518h36m30.378344595s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.496014314 (518h36m30.379427941s) [noise=true , address=89.149.201.147:4023 , piece_id=NQLJ6UGQTXWMNJ2XJSVQ2JPD2INWQNUPPQVHBYO6HPX6XX6HMWFA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.496680756 (518h36m30.380094372s) [noise=true , address=89.149.201.147:4006 , piece_id=JDGIM3TFHCOR7DT2EW4DHR4KKMYTBNJXEQQKI2YVB3OFIRXAMETA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.496801905 (518h36m30.380215532s) [noise=true , address=89.149.201.147:4019 , piece_id=3DMKAPD3M2A25PEKEWOOJCHV6CWKDUOZCZNPYDTQYBNSAML3MHCQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.497347115 (518h36m30.380760731s) [noise=true , address=89.149.201.147:4010 , piece_id=WCL5DLELLH7HWRF4NVXPG66W3PABG5LCQ7EI5QCHNGYL3WP26WSA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.497557301 (518h36m30.380970929s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498036118 (518h36m30.381449734s) [noise=true , address=89.149.201.147:4016 , piece_id=ZMUHQFGECGG6MKG7UWO4NWYOABOD6VYQ27JQNHSWGBCNY6RQOUCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498203330 (518h36m30.381616957s) [noise=true , address=185.167.204.205:4020 , piece_id=AF6L62IEU6KD2JIX7BL4VSIK6T7OCPETILSYGVGNS3MKX3XWTWKQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498564222 (518h36m30.381977836s) [noise=true , address=185.167.204.205:4023 , piece_id=B7OGRML6TDBWHQW7H5WZISNAEZLUTC4NUU6YJXLJG76ODSTTR2XA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498778308 (518h36m30.382191936s) [noise=true , address=81.171.14.70:4047 , piece_id=B2B7YV32YDVUG2L4Q6KV5FP63D6LIIVXEVWQEVQPLLO64NIWVK7A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.499060384 (518h36m30.382473998s) [noise=true , address=89.149.201.147:4017 , piece_id=OIUOJWKUHESMRGC2BHH45FNETDVKUMQTZHJICDYUDZCI66HS24EA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.499773462 (518h36m30.383187089s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.500485883 (518h36m30.383899502s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.500637044 (518h36m30.384050662s) [noise=true , address=89.149.201.147:4024 , piece_id=R6CF74YBNZ5NZHPCCBTUZ3PGHY56SAILYSNXV5L7FV37VORHXLZA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.500973891 (518h36m30.384387517s) [noise=true , address=81.171.14.70:4035 , piece_id=WMRQULXAT3UTOHK42S2ILFR7T7PICZUWCDRTMNOPCK2MKEZMI5LA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.501397526 (518h36m30.384811139s) [noise=true , address=89.149.201.147:4012 , piece_id=26IJ6NKKCUGV35XKHM4TISEX2CIWG3DNIHWYOHOMVMU4ANTCX6RA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.502337750 (518h36m30.385751362s) [noise=true , address=81.171.14.70:4030 , piece_id=FYBIQ4RINS5VAJS4VJKIJIREPWAZPESYAKSBKSZEX6OWXELKHJ3A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.502702474 (518h36m30.386116088s) [noise=true , address=81.171.14.70:4039 , piece_id=E4Q4U6JPWVZOWPDJI6A6AO7WQ72D2GOPMZQSYOHCWOOZVVE6NIKA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.503020550 (518h36m30.386434163s) [noise=true , address=81.171.14.70:4044 , piece_id=3XFKCWS4CN6C73YU54KJNACXZDBZUKBMSASVLAFGKOF5SGGB2UUA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.503356663 (518h36m30.386770277s) [noise=true , address=81.171.14.70:4031 , piece_id=GLUVAUI4ZVFE2C6OCNNRS3GVAFBJ6WG3ALYTNEBRSL6QBXA5B4LQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.504974620 (518h36m30.388388232s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.505290772 (518h36m30.388704384s) [noise=true , address=185.167.204.205:4027 , piece_id=4T7OLFAAULHVCZO7IOTX4CAJRA5J4ALHNB23B2SXPCSILECYTG6A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.505629943 (518h36m30.389043556s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.505933309 (518h36m30.389346921s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.507266987 (518h36m30.390680601s) [noise=true , address=89.149.201.147:4015 , piece_id=NPDGCLM2ZF64ZBOAPL2VT2XR4QMTFNZC3I43VVXLMGPCVER3F45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.507293406 (518h36m30.390707035s) [seg_pos={0 0}, upload=414748]: Segment reached optimal threshold of 26 pieces. Jul 14 23:47:30.507519964 (518h36m30.390933597s) [noise=true , address=81.171.14.70:4042 , piece_id=ASRUL4I2SLJEFFJLR6KBMGIAY3THJPR67YGCHBIMYNUFJKFRP2GA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD): protocol: expected piece hash; context canceled; EOF Jul 14 23:47:30.507609664 (518h36m30.391023293s) [noise=true , address=81.171.14.70:4029 , piece_id=LYKFSQWETSW6OJPHJYVD3LLP44S7RABUIBRHI5LMZSP2JJLDQIHQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to): protocol: expected piece hash; context canceled; EOF Jul 14 23:47:30.507634788 (518h36m30.391048419s) [seg_pos={0 0}, upload=414748]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Jul 14 23:47:30.507717732 (518h36m30.391131358s) [upload=414748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 14 23:47:30.507724547 (518h36m30.391138177s) [upload=414748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:47:30.547070340 (518h36m30.430483976s) [upload=414748]: Done uploading segments... Jul 14 23:47:30.547094478 (518h36m30.430508105s) [upload=414748]: Upload finished. err: <nil> Jul 14 23:47:30.863955222 (518h36m30.747368849s) [upload=414749]: Starting upload Jul 14 23:47:30.864023814 (518h36m30.747437456s) [upload=414749]: Done starting upload Jul 14 23:47:30.864126584 (518h36m30.747540202s) [upload=414749]: Uploading segments... Jul 14 23:47:30.864360135 (518h36m30.747773761s) [upload=414749]: Got next segment. Inline: false Jul 14 23:47:30.864426077 (518h36m30.747839704s) [upload=414749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:47:30.864431154 (518h36m30.747844779s) [upload=414749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 14 23:47:30.867084429 (518h36m30.750498058s) [upload=414749]: Next segment err: metaclient: Too Many Requests Jul 14 23:47:30.867113197 (518h36m30.750526824s) [upload=414749]: Done uploading segments... Jul 14 23:47:30.867126113 (518h36m30.750539725s) [upload=414749]: Upload finished. err: metaclient: Too Many Requests Duration: 389.761789ms core uplink eu-select-1 eu1-placement-15
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 550.963178ms FAIL (2025-07-14 23:49:40 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:54371->127.0.0.1:53: i/o timeout Duration: 551.005639ms edge authservice http bgp us1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [....................X.................................X.....] duration: 520.656281ms target: https://ap1.storj.io/login FAIL (2025-07-14 23:49:39 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp: lookup ap1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:60425->127.0.0.1:53: i/o timeout Duration: 520.679319ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.239147298s FAIL (2025-07-14 23:49:32 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.239173783s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 269.746771ms target: https://us1.storj.io/login FAIL (2025-07-14 23:49:32 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:55697->127.0.0.1:53: i/o timeout Duration: 269.76537ms core satellite us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 806.09657ms FAIL (2025-07-14 23:49:31 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:53438->127.0.0.1:53: i/o timeout Duration: 806.12911ms edge authservice http bgp ap1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X........X...] duration: 1.268824898s FAIL (2025-07-14 23:49:30 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.268872938s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 3 out of 60 is failed (threshold=3) [...........................................X..........XX....] duration: 17.388569ms target: https://eu1.storj.io/login FAIL (2025-07-14 23:49:30 UTC) Get "https://eu1.storj.io/login": dial tcp: lookup eu1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:36667->127.0.0.1:53: i/o timeout Duration: 17.411351ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 2.979410308s FAIL (2025-07-14 23:49:30 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:53438->127.0.0.1:53: i/o timeout Duration: 2.979432158s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 990.949853ms FAIL (2025-07-14 23:49:23 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load Jul 14 23:37:41.147633909 (518h26m41.031047542s) [upload=414628]: Done starting upload Jul 14 23:37:41.147700406 (518h26m41.031114026s) [upload=414628]: Uploading segments... Jul 14 23:37:41.147749349 (518h26m41.031162964s) [upload=414628]: Got next segment. Inline: true Jul 14 23:37:41.147757931 (518h26m41.031171548s) [upload=414628]: Waiting for error group managing segments... Jul 14 23:37:41.147803445 (518h26m41.031217064s) [upload=414628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:37:41.147810713 (518h26m41.031224329s) [upload=414628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:37:41.147814220 (518h26m41.031227838s) [upload=414628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:37:41.293868983 (518h26m41.177282609s) [upload=414628]: Done uploading segments... Jul 14 23:37:41.293899619 (518h26m41.177313246s) [upload=414628]: Upload finished. err: <nil> Jul 14 23:38:42.060547925 (518h27m41.943961552s) [upload=414641]: Starting upload Jul 14 23:38:42.060706705 (518h27m41.94412033s) [upload=414641]: Done starting upload Jul 14 23:38:42.060775681 (518h27m41.9441893s) [upload=414641]: Uploading segments... Jul 14 23:38:42.060850917 (518h27m41.944264562s) [upload=414641]: Got next segment. Inline: true Jul 14 23:38:42.060857684 (518h27m41.944271318s) [upload=414641]: Waiting for error group managing segments... Jul 14 23:38:42.060905522 (518h27m41.944319139s) [upload=414641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:38:42.060911671 (518h27m41.944325297s) [upload=414641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:38:42.060915517 (518h27m41.944329143s) [upload=414641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:38:42.214653360 (518h27m42.098066984s) [upload=414641]: Done uploading segments... Jul 14 23:38:42.214672288 (518h27m42.098085904s) [upload=414641]: Upload finished. err: <nil> Jul 14 23:39:37.897497561 (518h28m37.780911178s) [upload=414653]: Starting upload Jul 14 23:39:37.897653216 (518h28m37.781066839s) [upload=414653]: Done starting upload Jul 14 23:39:37.897770536 (518h28m37.781184151s) [upload=414653]: Uploading segments... Jul 14 23:39:37.897826735 (518h28m37.781240362s) [upload=414653]: Got next segment. Inline: true Jul 14 23:39:37.897832545 (518h28m37.78124616s) [upload=414653]: Waiting for error group managing segments... Jul 14 23:39:37.897874540 (518h28m37.781288156s) [upload=414653]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:39:37.897880498 (518h28m37.781294125s) [upload=414653]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:39:37.897884204 (518h28m37.78129783s) [upload=414653]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:39:38.174501392 (518h28m38.057915021s) [upload=414653]: Done uploading segments... Jul 14 23:39:38.174554097 (518h28m38.057967728s) [upload=414653]: Upload finished. err: <nil> Jul 14 23:40:47.713120393 (518h29m47.596534011s) [upload=414666]: Starting upload Jul 14 23:40:47.713181294 (518h29m47.596594929s) [upload=414666]: Done starting upload Jul 14 23:40:47.713333461 (518h29m47.596747074s) [upload=414666]: Uploading segments... Jul 14 23:40:47.713419846 (518h29m47.596833481s) [upload=414666]: Got next segment. Inline: true Jul 14 23:40:47.713427208 (518h29m47.596840876s) [upload=414666]: Waiting for error group managing segments... Jul 14 23:40:47.713492271 (518h29m47.596905915s) [upload=414666]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:40:47.713499814 (518h29m47.596913451s) [upload=414666]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:40:47.713503937 (518h29m47.596917586s) [upload=414666]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:40:47.868352520 (518h29m47.751766137s) [upload=414666]: Done uploading segments... Jul 14 23:40:47.868375465 (518h29m47.751789083s) [upload=414666]: Upload finished. err: <nil> Jul 14 23:41:00.797063324 (518h30m0.680476953s) [upload=414670]: Starting upload Jul 14 23:41:00.797141064 (518h30m0.68055469s) [upload=414670]: Done starting upload Jul 14 23:41:00.797212895 (518h30m0.680626511s) [upload=414670]: Uploading segments... Jul 14 23:41:00.797265690 (518h30m0.680679326s) [upload=414670]: Got next segment. Inline: true Jul 14 23:41:00.797270956 (518h30m0.680684572s) [upload=414670]: Waiting for error group managing segments... Jul 14 23:41:00.797328313 (518h30m0.680741939s) [upload=414670]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:41:00.797333853 (518h30m0.680747469s) [upload=414670]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:41:00.797337094 (518h30m0.68075072s) [upload=414670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:41:00.971853045 (518h30m0.855266664s) [upload=414670]: Done uploading segments... Jul 14 23:41:00.971870916 (518h30m0.855284535s) [upload=414670]: Upload finished. err: <nil> Jul 14 23:41:22.318635815 (518h30m22.202049441s) [upload=414675]: Starting upload Jul 14 23:41:22.318697752 (518h30m22.202111381s) [upload=414675]: Done starting upload Jul 14 23:41:22.318755348 (518h30m22.20216896s) [upload=414675]: Uploading segments... Jul 14 23:41:22.318804414 (518h30m22.202218058s) [upload=414675]: Got next segment. Inline: true Jul 14 23:41:22.318812150 (518h30m22.202225763s) [upload=414675]: Waiting for error group managing segments... Jul 14 23:41:22.318860496 (518h30m22.202274126s) [upload=414675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:41:22.318868059 (518h30m22.202281687s) [upload=414675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:41:22.318872383 (518h30m22.202286008s) [upload=414675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:41:22.468688511 (518h30m22.352102153s) [upload=414675]: Done uploading segments... Jul 14 23:41:22.468726173 (518h30m22.352139798s) [upload=414675]: Upload finished. err: <nil> Jul 14 23:42:39.877715214 (518h31m39.761128848s) [upload=414689]: Starting upload Jul 14 23:42:39.877790693 (518h31m39.76120433s) [upload=414689]: Done starting upload Jul 14 23:42:39.877900973 (518h31m39.761314587s) [upload=414689]: Uploading segments... Jul 14 23:42:39.878016572 (518h31m39.761430206s) [upload=414689]: Got next segment. Inline: true Jul 14 23:42:39.878023247 (518h31m39.761436881s) [upload=414689]: Waiting for error group managing segments... Jul 14 23:42:39.878073628 (518h31m39.761487264s) [upload=414689]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:42:39.878079532 (518h31m39.761493165s) [upload=414689]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:42:39.878083359 (518h31m39.761496973s) [upload=414689]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:42:40.027578980 (518h31m39.910992594s) [upload=414689]: Done uploading segments... Jul 14 23:42:40.027596526 (518h31m39.911010138s) [upload=414689]: Upload finished. err: <nil> Jul 14 23:42:40.961643522 (518h31m40.845057134s) [upload=414691]: Starting upload Jul 14 23:42:40.961719043 (518h31m40.845132672s) [upload=414691]: Done starting upload Jul 14 23:42:40.961792548 (518h31m40.84520616s) [upload=414691]: Uploading segments... Jul 14 23:42:40.961837433 (518h31m40.845251071s) [upload=414691]: Got next segment. Inline: true Jul 14 23:42:40.961851857 (518h31m40.845265486s) [upload=414691]: Waiting for error group managing segments... Jul 14 23:42:40.961890425 (518h31m40.845304037s) [upload=414691]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:42:40.961896055 (518h31m40.845309668s) [upload=414691]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:42:40.961898536 (518h31m40.845312147s) [upload=414691]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:42:41.122451811 (518h31m41.005865429s) [upload=414691]: Done uploading segments... Jul 14 23:42:41.122493314 (518h31m41.005906934s) [upload=414691]: Upload finished. err: <nil> Jul 14 23:43:34.049461829 (518h32m33.932875453s) [upload=414704]: Starting upload Jul 14 23:43:34.049588567 (518h32m33.933002206s) [upload=414704]: Done starting upload Jul 14 23:43:34.049735609 (518h32m33.933149224s) [upload=414704]: Uploading segments... Jul 14 23:43:34.049802945 (518h32m33.933216575s) [upload=414704]: Got next segment. Inline: true Jul 14 23:43:34.049810415 (518h32m33.933224046s) [upload=414704]: Waiting for error group managing segments... Jul 14 23:43:34.049877223 (518h32m33.93329084s) [upload=414704]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:43:34.049883389 (518h32m33.933297014s) [upload=414704]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:43:34.049887164 (518h32m33.933300789s) [upload=414704]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:43:34.197832086 (518h32m34.081245712s) [upload=414704]: Done uploading segments... Jul 14 23:43:34.197864776 (518h32m34.08127839s) [upload=414704]: Upload finished. err: <nil> Jul 14 23:43:56.290942463 (518h32m56.174356083s) [upload=414709]: Starting upload Jul 14 23:43:56.291049037 (518h32m56.174462655s) [upload=414709]: Done starting upload Jul 14 23:43:56.291143970 (518h32m56.174557592s) [upload=414709]: Uploading segments... Jul 14 23:43:56.291226174 (518h32m56.174639809s) [upload=414709]: Got next segment. Inline: true Jul 14 23:43:56.291231359 (518h32m56.174644972s) [upload=414709]: Waiting for error group managing segments... Jul 14 23:43:56.291289737 (518h32m56.17470335s) [upload=414709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:43:56.291295711 (518h32m56.174709323s) [upload=414709]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:43:56.291298840 (518h32m56.174712452s) [upload=414709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:43:56.428766698 (518h32m56.312180324s) [upload=414709]: Done uploading segments... Jul 14 23:43:56.428788768 (518h32m56.312202386s) [upload=414709]: Upload finished. err: <nil> Duration: 990.97566ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.682076011s FAIL (2025-07-14 23:49:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.68212017s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 326.129174ms target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-14 23:49:19 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp: lookup uptime-checks-us-select.s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:36040->127.0.0.1:53: i/o timeout Duration: 326.171338ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 428.86283ms FAIL (2025-07-14 23:49:13 UTC) orbiter warning: ======ERROR====== uplink: too many requests ======LOGS====== . Jul 14 23:47:30.334396460 (518h36m30.217810097s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.334541002 (518h36m30.217954628s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.334751321 (518h36m30.218164948s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.335060864 (518h36m30.218474493s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.335335126 (518h36m30.218748752s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.336922859 (518h36m30.220336495s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.340514312 (518h36m30.223927967s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.342052396 (518h36m30.225466052s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.485954867 (518h36m30.369368503s) [upload=414748]: Next returned nil segment Jul 14 23:47:30.485992786 (518h36m30.369406532s) [upload=414748]: Waiting for error group managing segments... Jul 14 23:47:30.494842357 (518h36m30.378255988s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.494930968 (518h36m30.378344595s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.496014314 (518h36m30.379427941s) [noise=true , address=89.149.201.147:4023 , piece_id=NQLJ6UGQTXWMNJ2XJSVQ2JPD2INWQNUPPQVHBYO6HPX6XX6HMWFA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.496680756 (518h36m30.380094372s) [noise=true , address=89.149.201.147:4006 , piece_id=JDGIM3TFHCOR7DT2EW4DHR4KKMYTBNJXEQQKI2YVB3OFIRXAMETA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.496801905 (518h36m30.380215532s) [noise=true , address=89.149.201.147:4019 , piece_id=3DMKAPD3M2A25PEKEWOOJCHV6CWKDUOZCZNPYDTQYBNSAML3MHCQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.497347115 (518h36m30.380760731s) [noise=true , address=89.149.201.147:4010 , piece_id=WCL5DLELLH7HWRF4NVXPG66W3PABG5LCQ7EI5QCHNGYL3WP26WSA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.497557301 (518h36m30.380970929s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498036118 (518h36m30.381449734s) [noise=true , address=89.149.201.147:4016 , piece_id=ZMUHQFGECGG6MKG7UWO4NWYOABOD6VYQ27JQNHSWGBCNY6RQOUCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498203330 (518h36m30.381616957s) [noise=true , address=185.167.204.205:4020 , piece_id=AF6L62IEU6KD2JIX7BL4VSIK6T7OCPETILSYGVGNS3MKX3XWTWKQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498564222 (518h36m30.381977836s) [noise=true , address=185.167.204.205:4023 , piece_id=B7OGRML6TDBWHQW7H5WZISNAEZLUTC4NUU6YJXLJG76ODSTTR2XA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498778308 (518h36m30.382191936s) [noise=true , address=81.171.14.70:4047 , piece_id=B2B7YV32YDVUG2L4Q6KV5FP63D6LIIVXEVWQEVQPLLO64NIWVK7A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.499060384 (518h36m30.382473998s) [noise=true , address=89.149.201.147:4017 , piece_id=OIUOJWKUHESMRGC2BHH45FNETDVKUMQTZHJICDYUDZCI66HS24EA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.499773462 (518h36m30.383187089s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.500485883 (518h36m30.383899502s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.500637044 (518h36m30.384050662s) [noise=true , address=89.149.201.147:4024 , piece_id=R6CF74YBNZ5NZHPCCBTUZ3PGHY56SAILYSNXV5L7FV37VORHXLZA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.500973891 (518h36m30.384387517s) [noise=true , address=81.171.14.70:4035 , piece_id=WMRQULXAT3UTOHK42S2ILFR7T7PICZUWCDRTMNOPCK2MKEZMI5LA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.501397526 (518h36m30.384811139s) [noise=true , address=89.149.201.147:4012 , piece_id=26IJ6NKKCUGV35XKHM4TISEX2CIWG3DNIHWYOHOMVMU4ANTCX6RA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.502337750 (518h36m30.385751362s) [noise=true , address=81.171.14.70:4030 , piece_id=FYBIQ4RINS5VAJS4VJKIJIREPWAZPESYAKSBKSZEX6OWXELKHJ3A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.502702474 (518h36m30.386116088s) [noise=true , address=81.171.14.70:4039 , piece_id=E4Q4U6JPWVZOWPDJI6A6AO7WQ72D2GOPMZQSYOHCWOOZVVE6NIKA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.503020550 (518h36m30.386434163s) [noise=true , address=81.171.14.70:4044 , piece_id=3XFKCWS4CN6C73YU54KJNACXZDBZUKBMSASVLAFGKOF5SGGB2UUA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.503356663 (518h36m30.386770277s) [noise=true , address=81.171.14.70:4031 , piece_id=GLUVAUI4ZVFE2C6OCNNRS3GVAFBJ6WG3ALYTNEBRSL6QBXA5B4LQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.504974620 (518h36m30.388388232s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.505290772 (518h36m30.388704384s) [noise=true , address=185.167.204.205:4027 , piece_id=4T7OLFAAULHVCZO7IOTX4CAJRA5J4ALHNB23B2SXPCSILECYTG6A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.505629943 (518h36m30.389043556s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.505933309 (518h36m30.389346921s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.507266987 (518h36m30.390680601s) [noise=true , address=89.149.201.147:4015 , piece_id=NPDGCLM2ZF64ZBOAPL2VT2XR4QMTFNZC3I43VVXLMGPCVER3F45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.507293406 (518h36m30.390707035s) [seg_pos={0 0}, upload=414748]: Segment reached optimal threshold of 26 pieces. Jul 14 23:47:30.507519964 (518h36m30.390933597s) [noise=true , address=81.171.14.70:4042 , piece_id=ASRUL4I2SLJEFFJLR6KBMGIAY3THJPR67YGCHBIMYNUFJKFRP2GA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD): protocol: expected piece hash; context canceled; EOF Jul 14 23:47:30.507609664 (518h36m30.391023293s) [noise=true , address=81.171.14.70:4029 , piece_id=LYKFSQWETSW6OJPHJYVD3LLP44S7RABUIBRHI5LMZSP2JJLDQIHQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to): protocol: expected piece hash; context canceled; EOF Jul 14 23:47:30.507634788 (518h36m30.391048419s) [seg_pos={0 0}, upload=414748]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Jul 14 23:47:30.507717732 (518h36m30.391131358s) [upload=414748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 14 23:47:30.507724547 (518h36m30.391138177s) [upload=414748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:47:30.547070340 (518h36m30.430483976s) [upload=414748]: Done uploading segments... Jul 14 23:47:30.547094478 (518h36m30.430508105s) [upload=414748]: Upload finished. err: <nil> Jul 14 23:47:30.863955222 (518h36m30.747368849s) [upload=414749]: Starting upload Jul 14 23:47:30.864023814 (518h36m30.747437456s) [upload=414749]: Done starting upload Jul 14 23:47:30.864126584 (518h36m30.747540202s) [upload=414749]: Uploading segments... Jul 14 23:47:30.864360135 (518h36m30.747773761s) [upload=414749]: Got next segment. Inline: false Jul 14 23:47:30.864426077 (518h36m30.747839704s) [upload=414749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:47:30.864431154 (518h36m30.747844779s) [upload=414749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 14 23:47:30.867084429 (518h36m30.750498058s) [upload=414749]: Next segment err: metaclient: Too Many Requests Jul 14 23:47:30.867113197 (518h36m30.750526824s) [upload=414749]: Done uploading segments... Jul 14 23:47:30.867126113 (518h36m30.750539725s) [upload=414749]: Upload finished. err: metaclient: Too Many Requests Duration: 428.918631ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 1.559575461s FAIL (2025-07-14 23:49:04 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.559604804s core uplink ap1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 784.049849ms FAIL (2025-07-14 23:49:04 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:53438->127.0.0.1:53: i/o timeout Duration: 784.083785ms edge authservice http bgp ap1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................X........X..] duration: 1.169278459s FAIL (2025-07-14 23:48:57 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.169315132s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 2.891436329s FAIL (2025-07-14 23:48:54 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:53438->127.0.0.1:53: i/o timeout Duration: 2.891463176s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.077141943s FAIL (2025-07-14 23:48:54 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/gateway-tests/from_do_prod_ops_orbiter_eu_0/1K?x-id=PutObject": dial tcp: lookup s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:57187->127.0.0.1:53: i/o timeout Duration: 1.077169753s edge gateway us-select-1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 3 out of 60 is failed (threshold=3) [............................................X..........XX...] duration: 9.392603ms target: https://eu1.storj.io/login FAIL (2025-07-14 23:48:48 UTC) Get "https://eu1.storj.io/login": dial tcp: lookup eu1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:36667->127.0.0.1:53: i/o timeout Duration: 9.418953ms core satellite eu1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 278.868348ms target: https://us1.storj.io/login FAIL (2025-07-14 23:48:47 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:55697->127.0.0.1:53: i/o timeout Duration: 278.891193ms core satellite us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.772592217s FAIL (2025-07-14 23:48:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.772612885s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 553.608194ms FAIL (2025-07-14 23:48:46 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:54371->127.0.0.1:53: i/o timeout Duration: 553.638637ms edge authservice http bgp us1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.....................X.................................X....] duration: 516.153672ms target: https://ap1.storj.io/login FAIL (2025-07-14 23:48:32 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp: lookup ap1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:60425->127.0.0.1:53: i/o timeout Duration: 516.247063ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.239802658s FAIL (2025-07-14 23:48:28 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.239927943s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.742885229s FAIL (2025-07-14 23:48:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.742903827s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 436.353459ms FAIL (2025-07-14 23:48:21 UTC) orbiter warning: ======ERROR====== uplink: too many requests ======LOGS====== . Jul 14 23:47:30.334396460 (518h36m30.217810097s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.334541002 (518h36m30.217954628s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.334751321 (518h36m30.218164948s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.335060864 (518h36m30.218474493s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.335335126 (518h36m30.218748752s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.336922859 (518h36m30.220336495s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.340514312 (518h36m30.223927967s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.342052396 (518h36m30.225466052s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Uploading piece... Jul 14 23:47:30.485954867 (518h36m30.369368503s) [upload=414748]: Next returned nil segment Jul 14 23:47:30.485992786 (518h36m30.369406532s) [upload=414748]: Waiting for error group managing segments... Jul 14 23:47:30.494842357 (518h36m30.378255988s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.494930968 (518h36m30.378344595s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.496014314 (518h36m30.379427941s) [noise=true , address=89.149.201.147:4023 , piece_id=NQLJ6UGQTXWMNJ2XJSVQ2JPD2INWQNUPPQVHBYO6HPX6XX6HMWFA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.496680756 (518h36m30.380094372s) [noise=true , address=89.149.201.147:4006 , piece_id=JDGIM3TFHCOR7DT2EW4DHR4KKMYTBNJXEQQKI2YVB3OFIRXAMETA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.496801905 (518h36m30.380215532s) [noise=true , address=89.149.201.147:4019 , piece_id=3DMKAPD3M2A25PEKEWOOJCHV6CWKDUOZCZNPYDTQYBNSAML3MHCQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.497347115 (518h36m30.380760731s) [noise=true , address=89.149.201.147:4010 , piece_id=WCL5DLELLH7HWRF4NVXPG66W3PABG5LCQ7EI5QCHNGYL3WP26WSA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.497557301 (518h36m30.380970929s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498036118 (518h36m30.381449734s) [noise=true , address=89.149.201.147:4016 , piece_id=ZMUHQFGECGG6MKG7UWO4NWYOABOD6VYQ27JQNHSWGBCNY6RQOUCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498203330 (518h36m30.381616957s) [noise=true , address=185.167.204.205:4020 , piece_id=AF6L62IEU6KD2JIX7BL4VSIK6T7OCPETILSYGVGNS3MKX3XWTWKQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498564222 (518h36m30.381977836s) [noise=true , address=185.167.204.205:4023 , piece_id=B7OGRML6TDBWHQW7H5WZISNAEZLUTC4NUU6YJXLJG76ODSTTR2XA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.498778308 (518h36m30.382191936s) [noise=true , address=81.171.14.70:4047 , piece_id=B2B7YV32YDVUG2L4Q6KV5FP63D6LIIVXEVWQEVQPLLO64NIWVK7A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.499060384 (518h36m30.382473998s) [noise=true , address=89.149.201.147:4017 , piece_id=OIUOJWKUHESMRGC2BHH45FNETDVKUMQTZHJICDYUDZCI66HS24EA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.499773462 (518h36m30.383187089s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.500485883 (518h36m30.383899502s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.500637044 (518h36m30.384050662s) [noise=true , address=89.149.201.147:4024 , piece_id=R6CF74YBNZ5NZHPCCBTUZ3PGHY56SAILYSNXV5L7FV37VORHXLZA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.500973891 (518h36m30.384387517s) [noise=true , address=81.171.14.70:4035 , piece_id=WMRQULXAT3UTOHK42S2ILFR7T7PICZUWCDRTMNOPCK2MKEZMI5LA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.501397526 (518h36m30.384811139s) [noise=true , address=89.149.201.147:4012 , piece_id=26IJ6NKKCUGV35XKHM4TISEX2CIWG3DNIHWYOHOMVMU4ANTCX6RA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.502337750 (518h36m30.385751362s) [noise=true , address=81.171.14.70:4030 , piece_id=FYBIQ4RINS5VAJS4VJKIJIREPWAZPESYAKSBKSZEX6OWXELKHJ3A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.502702474 (518h36m30.386116088s) [noise=true , address=81.171.14.70:4039 , piece_id=E4Q4U6JPWVZOWPDJI6A6AO7WQ72D2GOPMZQSYOHCWOOZVVE6NIKA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.503020550 (518h36m30.386434163s) [noise=true , address=81.171.14.70:4044 , piece_id=3XFKCWS4CN6C73YU54KJNACXZDBZUKBMSASVLAFGKOF5SGGB2UUA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.503356663 (518h36m30.386770277s) [noise=true , address=81.171.14.70:4031 , piece_id=GLUVAUI4ZVFE2C6OCNNRS3GVAFBJ6WG3ALYTNEBRSL6QBXA5B4LQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.504974620 (518h36m30.388388232s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.505290772 (518h36m30.388704384s) [noise=true , address=185.167.204.205:4027 , piece_id=4T7OLFAAULHVCZO7IOTX4CAJRA5J4ALHNB23B2SXPCSILECYTG6A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.505629943 (518h36m30.389043556s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.505933309 (518h36m30.389346921s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.507266987 (518h36m30.390680601s) [noise=true , address=89.149.201.147:4015 , piece_id=NPDGCLM2ZF64ZBOAPL2VT2XR4QMTFNZC3I43VVXLMGPCVER3F45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil> Jul 14 23:47:30.507293406 (518h36m30.390707035s) [seg_pos={0 0}, upload=414748]: Segment reached optimal threshold of 26 pieces. Jul 14 23:47:30.507519964 (518h36m30.390933597s) [noise=true , address=81.171.14.70:4042 , piece_id=ASRUL4I2SLJEFFJLR6KBMGIAY3THJPR67YGCHBIMYNUFJKFRP2GA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD): protocol: expected piece hash; context canceled; EOF Jul 14 23:47:30.507609664 (518h36m30.391023293s) [noise=true , address=81.171.14.70:4029 , piece_id=LYKFSQWETSW6OJPHJYVD3LLP44S7RABUIBRHI5LMZSP2JJLDQIHQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to): protocol: expected piece hash; context canceled; EOF Jul 14 23:47:30.507634788 (518h36m30.391048419s) [seg_pos={0 0}, upload=414748]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Jul 14 23:47:30.507717732 (518h36m30.391131358s) [upload=414748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 14 23:47:30.507724547 (518h36m30.391138177s) [upload=414748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:47:30.547070340 (518h36m30.430483976s) [upload=414748]: Done uploading segments... Jul 14 23:47:30.547094478 (518h36m30.430508105s) [upload=414748]: Upload finished. err: <nil> Jul 14 23:47:30.863955222 (518h36m30.747368849s) [upload=414749]: Starting upload Jul 14 23:47:30.864023814 (518h36m30.747437456s) [upload=414749]: Done starting upload Jul 14 23:47:30.864126584 (518h36m30.747540202s) [upload=414749]: Uploading segments... Jul 14 23:47:30.864360135 (518h36m30.747773761s) [upload=414749]: Got next segment. Inline: false Jul 14 23:47:30.864426077 (518h36m30.747839704s) [upload=414749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:47:30.864431154 (518h36m30.747844779s) [upload=414749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 14 23:47:30.867084429 (518h36m30.750498058s) [upload=414749]: Next segment err: metaclient: Too Many Requests Jul 14 23:47:30.867113197 (518h36m30.750526824s) [upload=414749]: Done uploading segments... Jul 14 23:47:30.867126113 (518h36m30.750539725s) [upload=414749]: Upload finished. err: metaclient: Too Many Requests Duration: 436.381514ms core uplink eu-select-1 eu1-placement-15
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 536.638903ms FAIL (2025-07-14 23:48:21 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:54371->127.0.0.1:53: i/o timeout Duration: 536.741011ms edge authservice http bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 914.663359ms FAIL (2025-07-14 23:48:15 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load Jul 14 23:37:41.147633909 (518h26m41.031047542s) [upload=414628]: Done starting upload Jul 14 23:37:41.147700406 (518h26m41.031114026s) [upload=414628]: Uploading segments... Jul 14 23:37:41.147749349 (518h26m41.031162964s) [upload=414628]: Got next segment. Inline: true Jul 14 23:37:41.147757931 (518h26m41.031171548s) [upload=414628]: Waiting for error group managing segments... Jul 14 23:37:41.147803445 (518h26m41.031217064s) [upload=414628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:37:41.147810713 (518h26m41.031224329s) [upload=414628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:37:41.147814220 (518h26m41.031227838s) [upload=414628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:37:41.293868983 (518h26m41.177282609s) [upload=414628]: Done uploading segments... Jul 14 23:37:41.293899619 (518h26m41.177313246s) [upload=414628]: Upload finished. err: <nil> Jul 14 23:38:42.060547925 (518h27m41.943961552s) [upload=414641]: Starting upload Jul 14 23:38:42.060706705 (518h27m41.94412033s) [upload=414641]: Done starting upload Jul 14 23:38:42.060775681 (518h27m41.9441893s) [upload=414641]: Uploading segments... Jul 14 23:38:42.060850917 (518h27m41.944264562s) [upload=414641]: Got next segment. Inline: true Jul 14 23:38:42.060857684 (518h27m41.944271318s) [upload=414641]: Waiting for error group managing segments... Jul 14 23:38:42.060905522 (518h27m41.944319139s) [upload=414641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:38:42.060911671 (518h27m41.944325297s) [upload=414641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:38:42.060915517 (518h27m41.944329143s) [upload=414641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:38:42.214653360 (518h27m42.098066984s) [upload=414641]: Done uploading segments... Jul 14 23:38:42.214672288 (518h27m42.098085904s) [upload=414641]: Upload finished. err: <nil> Jul 14 23:39:37.897497561 (518h28m37.780911178s) [upload=414653]: Starting upload Jul 14 23:39:37.897653216 (518h28m37.781066839s) [upload=414653]: Done starting upload Jul 14 23:39:37.897770536 (518h28m37.781184151s) [upload=414653]: Uploading segments... Jul 14 23:39:37.897826735 (518h28m37.781240362s) [upload=414653]: Got next segment. Inline: true Jul 14 23:39:37.897832545 (518h28m37.78124616s) [upload=414653]: Waiting for error group managing segments... Jul 14 23:39:37.897874540 (518h28m37.781288156s) [upload=414653]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:39:37.897880498 (518h28m37.781294125s) [upload=414653]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:39:37.897884204 (518h28m37.78129783s) [upload=414653]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:39:38.174501392 (518h28m38.057915021s) [upload=414653]: Done uploading segments... Jul 14 23:39:38.174554097 (518h28m38.057967728s) [upload=414653]: Upload finished. err: <nil> Jul 14 23:40:47.713120393 (518h29m47.596534011s) [upload=414666]: Starting upload Jul 14 23:40:47.713181294 (518h29m47.596594929s) [upload=414666]: Done starting upload Jul 14 23:40:47.713333461 (518h29m47.596747074s) [upload=414666]: Uploading segments... Jul 14 23:40:47.713419846 (518h29m47.596833481s) [upload=414666]: Got next segment. Inline: true Jul 14 23:40:47.713427208 (518h29m47.596840876s) [upload=414666]: Waiting for error group managing segments... Jul 14 23:40:47.713492271 (518h29m47.596905915s) [upload=414666]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:40:47.713499814 (518h29m47.596913451s) [upload=414666]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:40:47.713503937 (518h29m47.596917586s) [upload=414666]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:40:47.868352520 (518h29m47.751766137s) [upload=414666]: Done uploading segments... Jul 14 23:40:47.868375465 (518h29m47.751789083s) [upload=414666]: Upload finished. err: <nil> Jul 14 23:41:00.797063324 (518h30m0.680476953s) [upload=414670]: Starting upload Jul 14 23:41:00.797141064 (518h30m0.68055469s) [upload=414670]: Done starting upload Jul 14 23:41:00.797212895 (518h30m0.680626511s) [upload=414670]: Uploading segments... Jul 14 23:41:00.797265690 (518h30m0.680679326s) [upload=414670]: Got next segment. Inline: true Jul 14 23:41:00.797270956 (518h30m0.680684572s) [upload=414670]: Waiting for error group managing segments... Jul 14 23:41:00.797328313 (518h30m0.680741939s) [upload=414670]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:41:00.797333853 (518h30m0.680747469s) [upload=414670]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:41:00.797337094 (518h30m0.68075072s) [upload=414670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:41:00.971853045 (518h30m0.855266664s) [upload=414670]: Done uploading segments... Jul 14 23:41:00.971870916 (518h30m0.855284535s) [upload=414670]: Upload finished. err: <nil> Jul 14 23:41:22.318635815 (518h30m22.202049441s) [upload=414675]: Starting upload Jul 14 23:41:22.318697752 (518h30m22.202111381s) [upload=414675]: Done starting upload Jul 14 23:41:22.318755348 (518h30m22.20216896s) [upload=414675]: Uploading segments... Jul 14 23:41:22.318804414 (518h30m22.202218058s) [upload=414675]: Got next segment. Inline: true Jul 14 23:41:22.318812150 (518h30m22.202225763s) [upload=414675]: Waiting for error group managing segments... Jul 14 23:41:22.318860496 (518h30m22.202274126s) [upload=414675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:41:22.318868059 (518h30m22.202281687s) [upload=414675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:41:22.318872383 (518h30m22.202286008s) [upload=414675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:41:22.468688511 (518h30m22.352102153s) [upload=414675]: Done uploading segments... Jul 14 23:41:22.468726173 (518h30m22.352139798s) [upload=414675]: Upload finished. err: <nil> Jul 14 23:42:39.877715214 (518h31m39.761128848s) [upload=414689]: Starting upload Jul 14 23:42:39.877790693 (518h31m39.76120433s) [upload=414689]: Done starting upload Jul 14 23:42:39.877900973 (518h31m39.761314587s) [upload=414689]: Uploading segments... Jul 14 23:42:39.878016572 (518h31m39.761430206s) [upload=414689]: Got next segment. Inline: true Jul 14 23:42:39.878023247 (518h31m39.761436881s) [upload=414689]: Waiting for error group managing segments... Jul 14 23:42:39.878073628 (518h31m39.761487264s) [upload=414689]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:42:39.878079532 (518h31m39.761493165s) [upload=414689]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:42:39.878083359 (518h31m39.761496973s) [upload=414689]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:42:40.027578980 (518h31m39.910992594s) [upload=414689]: Done uploading segments... Jul 14 23:42:40.027596526 (518h31m39.911010138s) [upload=414689]: Upload finished. err: <nil> Jul 14 23:42:40.961643522 (518h31m40.845057134s) [upload=414691]: Starting upload Jul 14 23:42:40.961719043 (518h31m40.845132672s) [upload=414691]: Done starting upload Jul 14 23:42:40.961792548 (518h31m40.84520616s) [upload=414691]: Uploading segments... Jul 14 23:42:40.961837433 (518h31m40.845251071s) [upload=414691]: Got next segment. Inline: true Jul 14 23:42:40.961851857 (518h31m40.845265486s) [upload=414691]: Waiting for error group managing segments... Jul 14 23:42:40.961890425 (518h31m40.845304037s) [upload=414691]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:42:40.961896055 (518h31m40.845309668s) [upload=414691]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:42:40.961898536 (518h31m40.845312147s) [upload=414691]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:42:41.122451811 (518h31m41.005865429s) [upload=414691]: Done uploading segments... Jul 14 23:42:41.122493314 (518h31m41.005906934s) [upload=414691]: Upload finished. err: <nil> Jul 14 23:43:34.049461829 (518h32m33.932875453s) [upload=414704]: Starting upload Jul 14 23:43:34.049588567 (518h32m33.933002206s) [upload=414704]: Done starting upload Jul 14 23:43:34.049735609 (518h32m33.933149224s) [upload=414704]: Uploading segments... Jul 14 23:43:34.049802945 (518h32m33.933216575s) [upload=414704]: Got next segment. Inline: true Jul 14 23:43:34.049810415 (518h32m33.933224046s) [upload=414704]: Waiting for error group managing segments... Jul 14 23:43:34.049877223 (518h32m33.93329084s) [upload=414704]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:43:34.049883389 (518h32m33.933297014s) [upload=414704]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:43:34.049887164 (518h32m33.933300789s) [upload=414704]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:43:34.197832086 (518h32m34.081245712s) [upload=414704]: Done uploading segments... Jul 14 23:43:34.197864776 (518h32m34.08127839s) [upload=414704]: Upload finished. err: <nil> Jul 14 23:43:56.290942463 (518h32m56.174356083s) [upload=414709]: Starting upload Jul 14 23:43:56.291049037 (518h32m56.174462655s) [upload=414709]: Done starting upload Jul 14 23:43:56.291143970 (518h32m56.174557592s) [upload=414709]: Uploading segments... Jul 14 23:43:56.291226174 (518h32m56.174639809s) [upload=414709]: Got next segment. Inline: true Jul 14 23:43:56.291231359 (518h32m56.174644972s) [upload=414709]: Waiting for error group managing segments... Jul 14 23:43:56.291289737 (518h32m56.17470335s) [upload=414709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 14 23:43:56.291295711 (518h32m56.174709323s) [upload=414709]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 14 23:43:56.291298840 (518h32m56.174712452s) [upload=414709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 14 23:43:56.428766698 (518h32m56.312180324s) [upload=414709]: Done uploading segments... Jul 14 23:43:56.428788768 (518h32m56.312202386s) [upload=414709]: Upload finished. err: <nil> Duration: 914.690662ms core uplink us-select-1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 3 out of 60 is failed (threshold=3) [.............................................X..........XX..] duration: 21.686717ms target: https://eu1.storj.io/login FAIL (2025-07-14 23:48:14 UTC) Get "https://eu1.storj.io/login": dial tcp: lookup eu1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:36667->127.0.0.1:53: i/o timeout Duration: 21.72914ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 2.831397637s FAIL (2025-07-14 23:48:14 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:53438->127.0.0.1:53: i/o timeout Duration: 2.831423411s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 311.123402ms target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-14 23:48:13 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp: lookup uptime-checks-us-select.s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:36040->127.0.0.1:53: i/o timeout Duration: 311.147615ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.521964801s FAIL (2025-07-14 23:48:10 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.522004319s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.142957189s FAIL (2025-07-14 23:48:10 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/gateway-tests/from_do_prod_ops_orbiter_eu_0/1K?x-id=PutObject": dial tcp: lookup s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:57187->127.0.0.1:53: i/o timeout Duration: 1.142988834s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X........X.] duration: 1.190493577s FAIL (2025-07-14 23:48:04 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.190525738s core uplink eu1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 278.345762ms target: https://us1.storj.io/login FAIL (2025-07-14 23:47:59 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:55697->127.0.0.1:53: i/o timeout Duration: 278.369972ms core satellite us1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [......................X.................................X...] duration: 513.795577ms target: https://ap1.storj.io/login FAIL (2025-07-14 23:47:54 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp: lookup ap1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:60425->127.0.0.1:53: i/o timeout Duration: 513.881696ms core satellite ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.559399255s FAIL (2025-07-14 23:47:51 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.559425239s core uplink ap1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 751.694755ms FAIL (2025-07-14 23:47:49 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:53438->127.0.0.1:53: i/o timeout Duration: 751.733237ms edge authservice http bgp ap1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)