Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
267.944696ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-07-15 00:35:43 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:57172->127.0.0.1:53: i/o timeout
Duration: 267.971171ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
10.609033969s target:
https://us1.storj.io/login
FAIL (2025-07-15 00:35:33 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:58922->127.0.0.1:53: i/o timeout
Duration: 10.609061237s
core
satellite
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
46.711639ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-07-15 00:35:16 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41928->127.0.0.1:53: i/o timeout
Duration: 46.753096ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
9.67842ms target:
https://eu1.storj.io/login
FAIL (2025-07-15 00:35:09 UTC)
orbiter warning: Get "https://eu1.storj.io/login": dial tcp: lookup eu1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:57607->127.0.0.1:53: i/o timeout
Duration: 9.712244ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
886.110716ms
FAIL (2025-07-15 00:35:05 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 886.148902ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
203.866335ms
FAIL (2025-07-15 00:35:05 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 203.952577ms
core
uplink
eu1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
289.872182ms target:
https://us1.storj.io/login
FAIL (2025-07-15 00:35:03 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:58922->127.0.0.1:53: i/o timeout
Duration: 290.06853ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
117.144868ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-07-15 00:35:02 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:57172->127.0.0.1:53: i/o timeout
Duration: 117.184157ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.334786031s
FAIL (2025-07-15 00:35:00 UTC)
orbiter warning: uplink: failed to upload enough pieces (needed at least 49 but got 30)
Duration: 2.334876078s
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
177.014697ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
FAIL (2025-07-15 00:34:53 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:47171->127.0.0.1:53: i/o timeout
Duration: 177.039518ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
23.696205ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-07-15 00:34:51 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41928->127.0.0.1:53: i/o timeout
Duration: 23.73605ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
458.797258ms
FAIL (2025-07-15 00:34:43 UTC)
orbiter warning:
======ERROR======
uplink: too many requests
======LOGS======
.
Jul 14 23:47:30.334396460 (518h36m30.217810097s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.334541002 (518h36m30.217954628s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.334751321 (518h36m30.218164948s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.335060864 (518h36m30.218474493s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.335335126 (518h36m30.218748752s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.336922859 (518h36m30.220336495s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.340514312 (518h36m30.223927967s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.342052396 (518h36m30.225466052s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.485954867 (518h36m30.369368503s) [upload=414748]: Next returned nil segment
Jul 14 23:47:30.485992786 (518h36m30.369406532s) [upload=414748]: Waiting for error group managing segments...
Jul 14 23:47:30.494842357 (518h36m30.378255988s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.494930968 (518h36m30.378344595s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.496014314 (518h36m30.379427941s) [noise=true , address=89.149.201.147:4023 , piece_id=NQLJ6UGQTXWMNJ2XJSVQ2JPD2INWQNUPPQVHBYO6HPX6XX6HMWFA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.496680756 (518h36m30.380094372s) [noise=true , address=89.149.201.147:4006 , piece_id=JDGIM3TFHCOR7DT2EW4DHR4KKMYTBNJXEQQKI2YVB3OFIRXAMETA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.496801905 (518h36m30.380215532s) [noise=true , address=89.149.201.147:4019 , piece_id=3DMKAPD3M2A25PEKEWOOJCHV6CWKDUOZCZNPYDTQYBNSAML3MHCQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.497347115 (518h36m30.380760731s) [noise=true , address=89.149.201.147:4010 , piece_id=WCL5DLELLH7HWRF4NVXPG66W3PABG5LCQ7EI5QCHNGYL3WP26WSA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.497557301 (518h36m30.380970929s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498036118 (518h36m30.381449734s) [noise=true , address=89.149.201.147:4016 , piece_id=ZMUHQFGECGG6MKG7UWO4NWYOABOD6VYQ27JQNHSWGBCNY6RQOUCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498203330 (518h36m30.381616957s) [noise=true , address=185.167.204.205:4020 , piece_id=AF6L62IEU6KD2JIX7BL4VSIK6T7OCPETILSYGVGNS3MKX3XWTWKQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498564222 (518h36m30.381977836s) [noise=true , address=185.167.204.205:4023 , piece_id=B7OGRML6TDBWHQW7H5WZISNAEZLUTC4NUU6YJXLJG76ODSTTR2XA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498778308 (518h36m30.382191936s) [noise=true , address=81.171.14.70:4047 , piece_id=B2B7YV32YDVUG2L4Q6KV5FP63D6LIIVXEVWQEVQPLLO64NIWVK7A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.499060384 (518h36m30.382473998s) [noise=true , address=89.149.201.147:4017 , piece_id=OIUOJWKUHESMRGC2BHH45FNETDVKUMQTZHJICDYUDZCI66HS24EA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.499773462 (518h36m30.383187089s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.500485883 (518h36m30.383899502s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.500637044 (518h36m30.384050662s) [noise=true , address=89.149.201.147:4024 , piece_id=R6CF74YBNZ5NZHPCCBTUZ3PGHY56SAILYSNXV5L7FV37VORHXLZA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.500973891 (518h36m30.384387517s) [noise=true , address=81.171.14.70:4035 , piece_id=WMRQULXAT3UTOHK42S2ILFR7T7PICZUWCDRTMNOPCK2MKEZMI5LA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.501397526 (518h36m30.384811139s) [noise=true , address=89.149.201.147:4012 , piece_id=26IJ6NKKCUGV35XKHM4TISEX2CIWG3DNIHWYOHOMVMU4ANTCX6RA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.502337750 (518h36m30.385751362s) [noise=true , address=81.171.14.70:4030 , piece_id=FYBIQ4RINS5VAJS4VJKIJIREPWAZPESYAKSBKSZEX6OWXELKHJ3A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.502702474 (518h36m30.386116088s) [noise=true , address=81.171.14.70:4039 , piece_id=E4Q4U6JPWVZOWPDJI6A6AO7WQ72D2GOPMZQSYOHCWOOZVVE6NIKA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.503020550 (518h36m30.386434163s) [noise=true , address=81.171.14.70:4044 , piece_id=3XFKCWS4CN6C73YU54KJNACXZDBZUKBMSASVLAFGKOF5SGGB2UUA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.503356663 (518h36m30.386770277s) [noise=true , address=81.171.14.70:4031 , piece_id=GLUVAUI4ZVFE2C6OCNNRS3GVAFBJ6WG3ALYTNEBRSL6QBXA5B4LQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.504974620 (518h36m30.388388232s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.505290772 (518h36m30.388704384s) [noise=true , address=185.167.204.205:4027 , piece_id=4T7OLFAAULHVCZO7IOTX4CAJRA5J4ALHNB23B2SXPCSILECYTG6A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.505629943 (518h36m30.389043556s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.505933309 (518h36m30.389346921s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.507266987 (518h36m30.390680601s) [noise=true , address=89.149.201.147:4015 , piece_id=NPDGCLM2ZF64ZBOAPL2VT2XR4QMTFNZC3I43VVXLMGPCVER3F45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.507293406 (518h36m30.390707035s) [seg_pos={0 0}, upload=414748]: Segment reached optimal threshold of 26 pieces.
Jul 14 23:47:30.507519964 (518h36m30.390933597s) [noise=true , address=81.171.14.70:4042 , piece_id=ASRUL4I2SLJEFFJLR6KBMGIAY3THJPR67YGCHBIMYNUFJKFRP2GA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD): protocol: expected piece hash; context canceled; EOF
Jul 14 23:47:30.507609664 (518h36m30.391023293s) [noise=true , address=81.171.14.70:4029 , piece_id=LYKFSQWETSW6OJPHJYVD3LLP44S7RABUIBRHI5LMZSP2JJLDQIHQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to): protocol: expected piece hash; context canceled; EOF
Jul 14 23:47:30.507634788 (518h36m30.391048419s) [seg_pos={0 0}, upload=414748]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jul 14 23:47:30.507717732 (518h36m30.391131358s) [upload=414748]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 23:47:30.507724547 (518h36m30.391138177s) [upload=414748]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 23:47:30.547070340 (518h36m30.430483976s) [upload=414748]: Done uploading segments...
Jul 14 23:47:30.547094478 (518h36m30.430508105s) [upload=414748]: Upload finished. err: <nil>
Jul 14 23:47:30.863955222 (518h36m30.747368849s) [upload=414749]: Starting upload
Jul 14 23:47:30.864023814 (518h36m30.747437456s) [upload=414749]: Done starting upload
Jul 14 23:47:30.864126584 (518h36m30.747540202s) [upload=414749]: Uploading segments...
Jul 14 23:47:30.864360135 (518h36m30.747773761s) [upload=414749]: Got next segment. Inline: false
Jul 14 23:47:30.864426077 (518h36m30.747839704s) [upload=414749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 23:47:30.864431154 (518h36m30.747844779s) [upload=414749]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 14 23:47:30.867084429 (518h36m30.750498058s) [upload=414749]: Next segment err: metaclient: Too Many Requests
Jul 14 23:47:30.867113197 (518h36m30.750526824s) [upload=414749]: Done uploading segments...
Jul 14 23:47:30.867126113 (518h36m30.750539725s) [upload=414749]: Upload finished. err: metaclient: Too Many Requests
Duration: 458.831306ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
257.942077ms
FAIL (2025-07-15 00:34:43 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 258.051707ms
core
uplink
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
3 out of 60 is failed (threshold=3) [..................XX.............................X..........] duration:
291.165534ms target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-07-15 00:34:40 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp: lookup uptime-checks-us-select.s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:39508->127.0.0.1:53: i/o timeout
Duration: 291.191837ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
10.768051ms target:
https://eu1.storj.io/login
FAIL (2025-07-15 00:34:34 UTC)
orbiter warning: Get "https://eu1.storj.io/login": dial tcp: lookup eu1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:57607->127.0.0.1:53: i/o timeout
Duration: 10.872647ms
core
satellite
eu1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
271.3491ms target:
https://us1.storj.io/login
FAIL (2025-07-15 00:34:29 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:58922->127.0.0.1:53: i/o timeout
Duration: 271.382277ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
37.311594ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-07-15 00:34:19 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41928->127.0.0.1:53: i/o timeout
Duration: 37.351581ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
3 out of 60 is failed (threshold=3) [...................XX.............................X.........] duration:
284.923029ms target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-07-15 00:34:18 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp: lookup uptime-checks-us-select.s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:39508->127.0.0.1:53: i/o timeout
Duration: 284.988252ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
879.082781ms
FAIL (2025-07-15 00:34:16 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 879.112968ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
178.869165ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
FAIL (2025-07-15 00:34:14 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:47171->127.0.0.1:53: i/o timeout
Duration: 178.962056ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
10.070576ms target:
https://eu1.storj.io/login
FAIL (2025-07-15 00:34:12 UTC)
orbiter warning: Get "https://eu1.storj.io/login": dial tcp: lookup eu1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:57607->127.0.0.1:53: i/o timeout
Duration: 10.112411ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.456367214s
FAIL (2025-07-15 00:34:04 UTC)
orbiter warning: uplink: failed to upload enough pieces (needed at least 49 but got 30)
Duration: 2.456401032s
core
uplink
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
625.706563ms target:
https://us1.storj.io/login
FAIL (2025-07-15 00:33:58 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:58922->127.0.0.1:53: i/o timeout
Duration: 625.740798ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
116.21138ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-07-15 00:33:58 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:57172->127.0.0.1:53: i/o timeout
Duration: 116.236441ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
436.151807ms
FAIL (2025-07-15 00:33:55 UTC)
orbiter warning:
======ERROR======
uplink: too many requests
======LOGS======
.
Jul 14 23:47:30.334396460 (518h36m30.217810097s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.334541002 (518h36m30.217954628s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.334751321 (518h36m30.218164948s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.335060864 (518h36m30.218474493s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.335335126 (518h36m30.218748752s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.336922859 (518h36m30.220336495s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.340514312 (518h36m30.223927967s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.342052396 (518h36m30.225466052s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.485954867 (518h36m30.369368503s) [upload=414748]: Next returned nil segment
Jul 14 23:47:30.485992786 (518h36m30.369406532s) [upload=414748]: Waiting for error group managing segments...
Jul 14 23:47:30.494842357 (518h36m30.378255988s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.494930968 (518h36m30.378344595s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.496014314 (518h36m30.379427941s) [noise=true , address=89.149.201.147:4023 , piece_id=NQLJ6UGQTXWMNJ2XJSVQ2JPD2INWQNUPPQVHBYO6HPX6XX6HMWFA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.496680756 (518h36m30.380094372s) [noise=true , address=89.149.201.147:4006 , piece_id=JDGIM3TFHCOR7DT2EW4DHR4KKMYTBNJXEQQKI2YVB3OFIRXAMETA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.496801905 (518h36m30.380215532s) [noise=true , address=89.149.201.147:4019 , piece_id=3DMKAPD3M2A25PEKEWOOJCHV6CWKDUOZCZNPYDTQYBNSAML3MHCQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.497347115 (518h36m30.380760731s) [noise=true , address=89.149.201.147:4010 , piece_id=WCL5DLELLH7HWRF4NVXPG66W3PABG5LCQ7EI5QCHNGYL3WP26WSA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.497557301 (518h36m30.380970929s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498036118 (518h36m30.381449734s) [noise=true , address=89.149.201.147:4016 , piece_id=ZMUHQFGECGG6MKG7UWO4NWYOABOD6VYQ27JQNHSWGBCNY6RQOUCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498203330 (518h36m30.381616957s) [noise=true , address=185.167.204.205:4020 , piece_id=AF6L62IEU6KD2JIX7BL4VSIK6T7OCPETILSYGVGNS3MKX3XWTWKQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498564222 (518h36m30.381977836s) [noise=true , address=185.167.204.205:4023 , piece_id=B7OGRML6TDBWHQW7H5WZISNAEZLUTC4NUU6YJXLJG76ODSTTR2XA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498778308 (518h36m30.382191936s) [noise=true , address=81.171.14.70:4047 , piece_id=B2B7YV32YDVUG2L4Q6KV5FP63D6LIIVXEVWQEVQPLLO64NIWVK7A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.499060384 (518h36m30.382473998s) [noise=true , address=89.149.201.147:4017 , piece_id=OIUOJWKUHESMRGC2BHH45FNETDVKUMQTZHJICDYUDZCI66HS24EA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.499773462 (518h36m30.383187089s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.500485883 (518h36m30.383899502s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.500637044 (518h36m30.384050662s) [noise=true , address=89.149.201.147:4024 , piece_id=R6CF74YBNZ5NZHPCCBTUZ3PGHY56SAILYSNXV5L7FV37VORHXLZA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.500973891 (518h36m30.384387517s) [noise=true , address=81.171.14.70:4035 , piece_id=WMRQULXAT3UTOHK42S2ILFR7T7PICZUWCDRTMNOPCK2MKEZMI5LA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.501397526 (518h36m30.384811139s) [noise=true , address=89.149.201.147:4012 , piece_id=26IJ6NKKCUGV35XKHM4TISEX2CIWG3DNIHWYOHOMVMU4ANTCX6RA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.502337750 (518h36m30.385751362s) [noise=true , address=81.171.14.70:4030 , piece_id=FYBIQ4RINS5VAJS4VJKIJIREPWAZPESYAKSBKSZEX6OWXELKHJ3A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.502702474 (518h36m30.386116088s) [noise=true , address=81.171.14.70:4039 , piece_id=E4Q4U6JPWVZOWPDJI6A6AO7WQ72D2GOPMZQSYOHCWOOZVVE6NIKA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.503020550 (518h36m30.386434163s) [noise=true , address=81.171.14.70:4044 , piece_id=3XFKCWS4CN6C73YU54KJNACXZDBZUKBMSASVLAFGKOF5SGGB2UUA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.503356663 (518h36m30.386770277s) [noise=true , address=81.171.14.70:4031 , piece_id=GLUVAUI4ZVFE2C6OCNNRS3GVAFBJ6WG3ALYTNEBRSL6QBXA5B4LQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.504974620 (518h36m30.388388232s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.505290772 (518h36m30.388704384s) [noise=true , address=185.167.204.205:4027 , piece_id=4T7OLFAAULHVCZO7IOTX4CAJRA5J4ALHNB23B2SXPCSILECYTG6A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.505629943 (518h36m30.389043556s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.505933309 (518h36m30.389346921s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.507266987 (518h36m30.390680601s) [noise=true , address=89.149.201.147:4015 , piece_id=NPDGCLM2ZF64ZBOAPL2VT2XR4QMTFNZC3I43VVXLMGPCVER3F45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.507293406 (518h36m30.390707035s) [seg_pos={0 0}, upload=414748]: Segment reached optimal threshold of 26 pieces.
Jul 14 23:47:30.507519964 (518h36m30.390933597s) [noise=true , address=81.171.14.70:4042 , piece_id=ASRUL4I2SLJEFFJLR6KBMGIAY3THJPR67YGCHBIMYNUFJKFRP2GA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD): protocol: expected piece hash; context canceled; EOF
Jul 14 23:47:30.507609664 (518h36m30.391023293s) [noise=true , address=81.171.14.70:4029 , piece_id=LYKFSQWETSW6OJPHJYVD3LLP44S7RABUIBRHI5LMZSP2JJLDQIHQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to): protocol: expected piece hash; context canceled; EOF
Jul 14 23:47:30.507634788 (518h36m30.391048419s) [seg_pos={0 0}, upload=414748]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jul 14 23:47:30.507717732 (518h36m30.391131358s) [upload=414748]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 23:47:30.507724547 (518h36m30.391138177s) [upload=414748]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 23:47:30.547070340 (518h36m30.430483976s) [upload=414748]: Done uploading segments...
Jul 14 23:47:30.547094478 (518h36m30.430508105s) [upload=414748]: Upload finished. err: <nil>
Jul 14 23:47:30.863955222 (518h36m30.747368849s) [upload=414749]: Starting upload
Jul 14 23:47:30.864023814 (518h36m30.747437456s) [upload=414749]: Done starting upload
Jul 14 23:47:30.864126584 (518h36m30.747540202s) [upload=414749]: Uploading segments...
Jul 14 23:47:30.864360135 (518h36m30.747773761s) [upload=414749]: Got next segment. Inline: false
Jul 14 23:47:30.864426077 (518h36m30.747839704s) [upload=414749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 23:47:30.864431154 (518h36m30.747844779s) [upload=414749]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 14 23:47:30.867084429 (518h36m30.750498058s) [upload=414749]: Next segment err: metaclient: Too Many Requests
Jul 14 23:47:30.867113197 (518h36m30.750526824s) [upload=414749]: Done uploading segments...
Jul 14 23:47:30.867126113 (518h36m30.750539725s) [upload=414749]: Upload finished. err: metaclient: Too Many Requests
Duration: 436.20553ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
32.871393ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-07-15 00:33:54 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41928->127.0.0.1:53: i/o timeout
Duration: 32.900516ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
179.438754ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
FAIL (2025-07-15 00:33:54 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:47171->127.0.0.1:53: i/o timeout
Duration: 179.467749ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
881.330262ms
FAIL (2025-07-15 00:33:35 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 881.361542ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
71.261434ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-07-15 00:33:33 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41928->127.0.0.1:53: i/o timeout
Duration: 71.288869ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
203.662112ms
FAIL (2025-07-15 00:33:31 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 203.693332ms
core
uplink
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
3 out of 60 is failed (threshold=3) [....................XX.............................X........] duration:
352.809459ms target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-07-15 00:33:31 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp: lookup uptime-checks-us-select.s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:39508->127.0.0.1:53: i/o timeout
Duration: 352.836016ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
135.926226ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-07-15 00:33:30 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:57172->127.0.0.1:53: i/o timeout
Duration: 135.973666ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
180.581937ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
FAIL (2025-07-15 00:33:19 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:47171->127.0.0.1:53: i/o timeout
Duration: 180.607601ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
1.974100145s target:
https://us1.storj.io/login
FAIL (2025-07-15 00:33:16 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:58922->127.0.0.1:53: i/o timeout
Duration: 1.974125786s
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
923.014651ms
FAIL (2025-07-15 00:33:08 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 923.044537ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
200.299166ms
FAIL (2025-07-15 00:32:59 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 200.341108ms
core
uplink
eu1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [X........X..................................................] duration:
8.874825ms target:
https://eu1.storj.io/login
FAIL (2025-07-15 00:32:58 UTC)
orbiter warning: Get "https://eu1.storj.io/login": dial tcp: lookup eu1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:57607->127.0.0.1:53: i/o timeout
Duration: 8.917436ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.50315439s
FAIL (2025-07-15 00:32:54 UTC)
orbiter warning: uplink: failed to upload enough pieces (needed at least 49 but got 30)
Duration: 2.503187194s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
447.053628ms
FAIL (2025-07-15 00:32:50 UTC)
orbiter warning:
======ERROR======
uplink: too many requests
======LOGS======
.
Jul 14 23:47:30.334396460 (518h36m30.217810097s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.334541002 (518h36m30.217954628s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.334751321 (518h36m30.218164948s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.335060864 (518h36m30.218474493s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.335335126 (518h36m30.218748752s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.336922859 (518h36m30.220336495s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.340514312 (518h36m30.223927967s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.342052396 (518h36m30.225466052s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.485954867 (518h36m30.369368503s) [upload=414748]: Next returned nil segment
Jul 14 23:47:30.485992786 (518h36m30.369406532s) [upload=414748]: Waiting for error group managing segments...
Jul 14 23:47:30.494842357 (518h36m30.378255988s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.494930968 (518h36m30.378344595s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.496014314 (518h36m30.379427941s) [noise=true , address=89.149.201.147:4023 , piece_id=NQLJ6UGQTXWMNJ2XJSVQ2JPD2INWQNUPPQVHBYO6HPX6XX6HMWFA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.496680756 (518h36m30.380094372s) [noise=true , address=89.149.201.147:4006 , piece_id=JDGIM3TFHCOR7DT2EW4DHR4KKMYTBNJXEQQKI2YVB3OFIRXAMETA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.496801905 (518h36m30.380215532s) [noise=true , address=89.149.201.147:4019 , piece_id=3DMKAPD3M2A25PEKEWOOJCHV6CWKDUOZCZNPYDTQYBNSAML3MHCQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.497347115 (518h36m30.380760731s) [noise=true , address=89.149.201.147:4010 , piece_id=WCL5DLELLH7HWRF4NVXPG66W3PABG5LCQ7EI5QCHNGYL3WP26WSA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.497557301 (518h36m30.380970929s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498036118 (518h36m30.381449734s) [noise=true , address=89.149.201.147:4016 , piece_id=ZMUHQFGECGG6MKG7UWO4NWYOABOD6VYQ27JQNHSWGBCNY6RQOUCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498203330 (518h36m30.381616957s) [noise=true , address=185.167.204.205:4020 , piece_id=AF6L62IEU6KD2JIX7BL4VSIK6T7OCPETILSYGVGNS3MKX3XWTWKQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498564222 (518h36m30.381977836s) [noise=true , address=185.167.204.205:4023 , piece_id=B7OGRML6TDBWHQW7H5WZISNAEZLUTC4NUU6YJXLJG76ODSTTR2XA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498778308 (518h36m30.382191936s) [noise=true , address=81.171.14.70:4047 , piece_id=B2B7YV32YDVUG2L4Q6KV5FP63D6LIIVXEVWQEVQPLLO64NIWVK7A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.499060384 (518h36m30.382473998s) [noise=true , address=89.149.201.147:4017 , piece_id=OIUOJWKUHESMRGC2BHH45FNETDVKUMQTZHJICDYUDZCI66HS24EA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.499773462 (518h36m30.383187089s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.500485883 (518h36m30.383899502s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.500637044 (518h36m30.384050662s) [noise=true , address=89.149.201.147:4024 , piece_id=R6CF74YBNZ5NZHPCCBTUZ3PGHY56SAILYSNXV5L7FV37VORHXLZA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.500973891 (518h36m30.384387517s) [noise=true , address=81.171.14.70:4035 , piece_id=WMRQULXAT3UTOHK42S2ILFR7T7PICZUWCDRTMNOPCK2MKEZMI5LA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.501397526 (518h36m30.384811139s) [noise=true , address=89.149.201.147:4012 , piece_id=26IJ6NKKCUGV35XKHM4TISEX2CIWG3DNIHWYOHOMVMU4ANTCX6RA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.502337750 (518h36m30.385751362s) [noise=true , address=81.171.14.70:4030 , piece_id=FYBIQ4RINS5VAJS4VJKIJIREPWAZPESYAKSBKSZEX6OWXELKHJ3A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.502702474 (518h36m30.386116088s) [noise=true , address=81.171.14.70:4039 , piece_id=E4Q4U6JPWVZOWPDJI6A6AO7WQ72D2GOPMZQSYOHCWOOZVVE6NIKA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.503020550 (518h36m30.386434163s) [noise=true , address=81.171.14.70:4044 , piece_id=3XFKCWS4CN6C73YU54KJNACXZDBZUKBMSASVLAFGKOF5SGGB2UUA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.503356663 (518h36m30.386770277s) [noise=true , address=81.171.14.70:4031 , piece_id=GLUVAUI4ZVFE2C6OCNNRS3GVAFBJ6WG3ALYTNEBRSL6QBXA5B4LQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.504974620 (518h36m30.388388232s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.505290772 (518h36m30.388704384s) [noise=true , address=185.167.204.205:4027 , piece_id=4T7OLFAAULHVCZO7IOTX4CAJRA5J4ALHNB23B2SXPCSILECYTG6A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.505629943 (518h36m30.389043556s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.505933309 (518h36m30.389346921s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.507266987 (518h36m30.390680601s) [noise=true , address=89.149.201.147:4015 , piece_id=NPDGCLM2ZF64ZBOAPL2VT2XR4QMTFNZC3I43VVXLMGPCVER3F45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.507293406 (518h36m30.390707035s) [seg_pos={0 0}, upload=414748]: Segment reached optimal threshold of 26 pieces.
Jul 14 23:47:30.507519964 (518h36m30.390933597s) [noise=true , address=81.171.14.70:4042 , piece_id=ASRUL4I2SLJEFFJLR6KBMGIAY3THJPR67YGCHBIMYNUFJKFRP2GA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD): protocol: expected piece hash; context canceled; EOF
Jul 14 23:47:30.507609664 (518h36m30.391023293s) [noise=true , address=81.171.14.70:4029 , piece_id=LYKFSQWETSW6OJPHJYVD3LLP44S7RABUIBRHI5LMZSP2JJLDQIHQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to): protocol: expected piece hash; context canceled; EOF
Jul 14 23:47:30.507634788 (518h36m30.391048419s) [seg_pos={0 0}, upload=414748]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jul 14 23:47:30.507717732 (518h36m30.391131358s) [upload=414748]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 23:47:30.507724547 (518h36m30.391138177s) [upload=414748]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 23:47:30.547070340 (518h36m30.430483976s) [upload=414748]: Done uploading segments...
Jul 14 23:47:30.547094478 (518h36m30.430508105s) [upload=414748]: Upload finished. err: <nil>
Jul 14 23:47:30.863955222 (518h36m30.747368849s) [upload=414749]: Starting upload
Jul 14 23:47:30.864023814 (518h36m30.747437456s) [upload=414749]: Done starting upload
Jul 14 23:47:30.864126584 (518h36m30.747540202s) [upload=414749]: Uploading segments...
Jul 14 23:47:30.864360135 (518h36m30.747773761s) [upload=414749]: Got next segment. Inline: false
Jul 14 23:47:30.864426077 (518h36m30.747839704s) [upload=414749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 23:47:30.864431154 (518h36m30.747844779s) [upload=414749]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 14 23:47:30.867084429 (518h36m30.750498058s) [upload=414749]: Next segment err: metaclient: Too Many Requests
Jul 14 23:47:30.867113197 (518h36m30.750526824s) [upload=414749]: Done uploading segments...
Jul 14 23:47:30.867126113 (518h36m30.750539725s) [upload=414749]: Upload finished. err: metaclient: Too Many Requests
Duration: 447.0916ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
105.814723ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-07-15 00:32:42 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:57172->127.0.0.1:53: i/o timeout
Duration: 105.838175ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
28.804401ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-07-15 00:32:42 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41928->127.0.0.1:53: i/o timeout
Duration: 28.828908ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
182.087867ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
FAIL (2025-07-15 00:32:36 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:47171->127.0.0.1:53: i/o timeout
Duration: 182.114322ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
400.388822ms
FAIL (2025-07-15 00:32:25 UTC)
orbiter warning:
======ERROR======
uplink: too many requests
======LOGS======
.
Jul 14 23:47:30.334396460 (518h36m30.217810097s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.334541002 (518h36m30.217954628s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.334751321 (518h36m30.218164948s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.335060864 (518h36m30.218474493s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.335335126 (518h36m30.218748752s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.336922859 (518h36m30.220336495s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.340514312 (518h36m30.223927967s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.342052396 (518h36m30.225466052s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Uploading piece...
Jul 14 23:47:30.485954867 (518h36m30.369368503s) [upload=414748]: Next returned nil segment
Jul 14 23:47:30.485992786 (518h36m30.369406532s) [upload=414748]: Waiting for error group managing segments...
Jul 14 23:47:30.494842357 (518h36m30.378255988s) [noise=true , address=89.149.201.147:4005 , piece_id=DRCMTHMTAA3GDMFIOJX2SHSFA3TL7KC2U3CJ3NNUEQQXGCW5PGSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.494930968 (518h36m30.378344595s) [noise=true , address=89.149.201.147:4022 , piece_id=7ZS6QYRU5CQZZ3KHH3TVBLGWT6G4HFVGUDEDYTXQ6DWYIQAISMLQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.496014314 (518h36m30.379427941s) [noise=true , address=89.149.201.147:4023 , piece_id=NQLJ6UGQTXWMNJ2XJSVQ2JPD2INWQNUPPQVHBYO6HPX6XX6HMWFA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.496680756 (518h36m30.380094372s) [noise=true , address=89.149.201.147:4006 , piece_id=JDGIM3TFHCOR7DT2EW4DHR4KKMYTBNJXEQQKI2YVB3OFIRXAMETA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.496801905 (518h36m30.380215532s) [noise=true , address=89.149.201.147:4019 , piece_id=3DMKAPD3M2A25PEKEWOOJCHV6CWKDUOZCZNPYDTQYBNSAML3MHCQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.497347115 (518h36m30.380760731s) [noise=true , address=89.149.201.147:4010 , piece_id=WCL5DLELLH7HWRF4NVXPG66W3PABG5LCQ7EI5QCHNGYL3WP26WSA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.497557301 (518h36m30.380970929s) [noise=true , address=81.171.14.70:4027 , piece_id=UFITELK6U7CJ5MO6DK46KMDBADRXL4X3ITLBBGPDIKONOIU2ZV6Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498036118 (518h36m30.381449734s) [noise=true , address=89.149.201.147:4016 , piece_id=ZMUHQFGECGG6MKG7UWO4NWYOABOD6VYQ27JQNHSWGBCNY6RQOUCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498203330 (518h36m30.381616957s) [noise=true , address=185.167.204.205:4020 , piece_id=AF6L62IEU6KD2JIX7BL4VSIK6T7OCPETILSYGVGNS3MKX3XWTWKQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498564222 (518h36m30.381977836s) [noise=true , address=185.167.204.205:4023 , piece_id=B7OGRML6TDBWHQW7H5WZISNAEZLUTC4NUU6YJXLJG76ODSTTR2XA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.498778308 (518h36m30.382191936s) [noise=true , address=81.171.14.70:4047 , piece_id=B2B7YV32YDVUG2L4Q6KV5FP63D6LIIVXEVWQEVQPLLO64NIWVK7A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.499060384 (518h36m30.382473998s) [noise=true , address=89.149.201.147:4017 , piece_id=OIUOJWKUHESMRGC2BHH45FNETDVKUMQTZHJICDYUDZCI66HS24EA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.499773462 (518h36m30.383187089s) [noise=true , address=185.167.204.206:4056 , piece_id=VYVMHTMO2RWAXNUCUZRMEUH4ONO67QDRV4F5M57BGNX35LYXB45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.500485883 (518h36m30.383899502s) [noise=true , address=81.171.14.70:4033 , piece_id=D4AOU75QAT7HSFOS3CCHDE5MPT4W6DCX5FZD4HFKFRVL3ZNWQMSQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.500637044 (518h36m30.384050662s) [noise=true , address=89.149.201.147:4024 , piece_id=R6CF74YBNZ5NZHPCCBTUZ3PGHY56SAILYSNXV5L7FV37VORHXLZA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.500973891 (518h36m30.384387517s) [noise=true , address=81.171.14.70:4035 , piece_id=WMRQULXAT3UTOHK42S2ILFR7T7PICZUWCDRTMNOPCK2MKEZMI5LA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.501397526 (518h36m30.384811139s) [noise=true , address=89.149.201.147:4012 , piece_id=26IJ6NKKCUGV35XKHM4TISEX2CIWG3DNIHWYOHOMVMU4ANTCX6RA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.502337750 (518h36m30.385751362s) [noise=true , address=81.171.14.70:4030 , piece_id=FYBIQ4RINS5VAJS4VJKIJIREPWAZPESYAKSBKSZEX6OWXELKHJ3A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.502702474 (518h36m30.386116088s) [noise=true , address=81.171.14.70:4039 , piece_id=E4Q4U6JPWVZOWPDJI6A6AO7WQ72D2GOPMZQSYOHCWOOZVVE6NIKA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.503020550 (518h36m30.386434163s) [noise=true , address=81.171.14.70:4044 , piece_id=3XFKCWS4CN6C73YU54KJNACXZDBZUKBMSASVLAFGKOF5SGGB2UUA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.503356663 (518h36m30.386770277s) [noise=true , address=81.171.14.70:4031 , piece_id=GLUVAUI4ZVFE2C6OCNNRS3GVAFBJ6WG3ALYTNEBRSL6QBXA5B4LQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.504974620 (518h36m30.388388232s) [noise=true , address=185.167.204.206:4060 , piece_id=IJQZL4K3DLVPFO456GG4WUGUOPKCHMZRQ42RDWGBKYC5XEQU7Y3Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.505290772 (518h36m30.388704384s) [noise=true , address=185.167.204.205:4027 , piece_id=4T7OLFAAULHVCZO7IOTX4CAJRA5J4ALHNB23B2SXPCSILECYTG6A, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.505629943 (518h36m30.389043556s) [noise=true , address=185.167.204.205:4035 , piece_id=NGLYETXCGUBW4B6TUTQFZXSRADNS2XPDMIU2SQXDAKHUHOT6XHDA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.505933309 (518h36m30.389346921s) [noise=true , address=185.167.204.205:4036 , piece_id=BHX33VXHOFYBWTEXGC6PP2VODAWV4PUD5E2VH3FKS7Z3RXTRFJCA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.507266987 (518h36m30.390680601s) [noise=true , address=89.149.201.147:4015 , piece_id=NPDGCLM2ZF64ZBOAPL2VT2XR4QMTFNZC3I43VVXLMGPCVER3F45Q, seg_pos={0 0}, upload=414748]: Done uploading piece. err: <nil>
Jul 14 23:47:30.507293406 (518h36m30.390707035s) [seg_pos={0 0}, upload=414748]: Segment reached optimal threshold of 26 pieces.
Jul 14 23:47:30.507519964 (518h36m30.390933597s) [noise=true , address=81.171.14.70:4042 , piece_id=ASRUL4I2SLJEFFJLR6KBMGIAY3THJPR67YGCHBIMYNUFJKFRP2GA, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD): protocol: expected piece hash; context canceled; EOF
Jul 14 23:47:30.507609664 (518h36m30.391023293s) [noise=true , address=81.171.14.70:4029 , piece_id=LYKFSQWETSW6OJPHJYVD3LLP44S7RABUIBRHI5LMZSP2JJLDQIHQ, seg_pos={0 0}, upload=414748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to): protocol: expected piece hash; context canceled; EOF
Jul 14 23:47:30.507634788 (518h36m30.391048419s) [seg_pos={0 0}, upload=414748]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jul 14 23:47:30.507717732 (518h36m30.391131358s) [upload=414748]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 23:47:30.507724547 (518h36m30.391138177s) [upload=414748]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 23:47:30.547070340 (518h36m30.430483976s) [upload=414748]: Done uploading segments...
Jul 14 23:47:30.547094478 (518h36m30.430508105s) [upload=414748]: Upload finished. err: <nil>
Jul 14 23:47:30.863955222 (518h36m30.747368849s) [upload=414749]: Starting upload
Jul 14 23:47:30.864023814 (518h36m30.747437456s) [upload=414749]: Done starting upload
Jul 14 23:47:30.864126584 (518h36m30.747540202s) [upload=414749]: Uploading segments...
Jul 14 23:47:30.864360135 (518h36m30.747773761s) [upload=414749]: Got next segment. Inline: false
Jul 14 23:47:30.864426077 (518h36m30.747839704s) [upload=414749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 23:47:30.864431154 (518h36m30.747844779s) [upload=414749]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 14 23:47:30.867084429 (518h36m30.750498058s) [upload=414749]: Next segment err: metaclient: Too Many Requests
Jul 14 23:47:30.867113197 (518h36m30.750526824s) [upload=414749]: Done uploading segments...
Jul 14 23:47:30.867126113 (518h36m30.750539725s) [upload=414749]: Upload finished. err: metaclient: Too Many Requests
Duration: 400.436963ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
3 out of 60 is failed (threshold=3) [XX........X.................................................] duration:
12.039345ms target:
https://eu1.storj.io/login
FAIL (2025-07-15 00:32:24 UTC)
orbiter warning: Get "https://eu1.storj.io/login": dial tcp: lookup eu1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:57607->127.0.0.1:53: i/o timeout
Duration: 12.099604ms
core
satellite
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
3 out of 60 is failed (threshold=3) [.....................XX.............................X.......] duration:
340.526814ms target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-07-15 00:32:17 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp: lookup uptime-checks-us-select.s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:39508->127.0.0.1:53: i/o timeout
Duration: 340.557947ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
856.284005ms
FAIL (2025-07-15 00:32:16 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 856.440084ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
177.834438ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
FAIL (2025-07-15 00:32:14 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:47171->127.0.0.1:53: i/o timeout
Duration: 177.857695ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
20.088562ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-07-15 00:32:08 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41928->127.0.0.1:53: i/o timeout
Duration: 20.113736ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.376432112s
FAIL (2025-07-15 00:32:07 UTC)
orbiter warning: uplink: failed to upload enough pieces (needed at least 49 but got 30)
Duration: 2.376462672s
core
uplink
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
273.407336ms target:
https://us1.storj.io/login
FAIL (2025-07-15 00:32:04 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:58922->127.0.0.1:53: i/o timeout
Duration: 273.463968ms
core
satellite
us1
|