Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.572277331s
FAIL (2025-09-11 20:45:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.572392481s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.611679147s
FAIL (2025-09-11 20:44:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.611711227s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.711976172s
FAIL (2025-09-11 20:43:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.712099096s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.519115188s
FAIL (2025-09-11 20:42:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.519152854s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.673951583s
FAIL (2025-09-11 20:41:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.67404487s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.788642443s
FAIL (2025-09-11 20:41:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.788677015s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.790964458s
FAIL (2025-09-11 20:40:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.790993551s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
2.652356362s
FAIL (2025-09-11 20:40:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.652392356s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
2.756898189s
FAIL (2025-09-11 20:39:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.757016939s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.619421753s
FAIL (2025-09-11 20:38:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.619551355s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.882311917s
FAIL (2025-09-11 20:37:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.882336168s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.974809061s
FAIL (2025-09-11 20:36:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.974830995s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.000708391s
FAIL (2025-09-11 20:36:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000737961s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
931.910768ms
FAIL (2025-09-11 05:47:47 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 931.963002ms
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
340.617762ms target:
https://version.storj.io/
FAIL (2025-09-11 05:47:08 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 340.642652ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
946.12329ms
FAIL (2025-09-11 05:46:44 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 946.156296ms
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................] duration:
341.197783ms target:
https://version.storj.io/
FAIL (2025-09-11 05:46:15 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 341.231407ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
860.701976ms
FAIL (2025-09-11 05:45:51 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 860.735042ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.493456024s
FAIL (2025-09-11 05:45:49 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 2.493498725s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
2 out of 60 is failed (threshold=3) [.XX.........................................................] duration:
332.25787ms target:
https://version.storj.io/
FAIL (2025-09-11 05:45:35 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 332.285227ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
863.635534ms
FAIL (2025-09-11 05:45:30 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 863.74412ms
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
2 out of 60 is failed (threshold=3) [..XX........................................................] duration:
329.828596ms target:
https://version.storj.io/
FAIL (2025-09-11 05:45:12 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 329.90487ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
882.96421ms
FAIL (2025-09-11 05:44:55 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 882.997094ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................] duration:
2.411365074s
FAIL (2025-09-11 05:44:51 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 2.411394673s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
2 out of 60 is failed (threshold=3) [...XX.......................................................] duration:
382.91332ms target:
https://version.storj.io/
FAIL (2025-09-11 05:44:33 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 382.949724ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.XX.........................................................] duration:
2.293009084s
FAIL (2025-09-11 05:44:30 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 2.293064801s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.396691912s
FAIL (2025-09-11 05:44:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
6387]: Got next segment. Inline: false
Sep 11 05:01:47.070094760 (302h54m7.562527201s) [upload=266387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 11 05:01:47.070100812 (302h54m7.562533255s) [upload=266387]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 11 05:01:47.282517324 (302h54m7.774949756s) [seg_pos={0 0}, upload=266387]: Begin upload segment...
Sep 11 05:01:47.282586917 (302h54m7.775019351s) [seg_pos={0 0}, upload=266387]: Done begin upload segment.
Sep 11 05:01:47.282630874 (302h54m7.775063319s) [noise=true , address=145.79.227.2:4121 , piece_id=KUVJMQVN5VOZCGW5W4DY5BMLI5IH23GSC2IFN6PH6XEA4OT3VUMA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283078686 (302h54m7.775511133s) [noise=true , address=45.91.116.8:4051 , piece_id=PPCINK7K25XXRN3JQPMWVBQVPXXP7UPVSILUGIKWO3OI7EEI42FQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283596723 (302h54m7.776029168s) [noise=true , address=45.91.116.11:4147 , piece_id=BPNZTKMMEXM5B6UQSOLWZTZVVT5UOV2QHPSZL742DHQ3AQ3O6ESA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283956314 (302h54m7.776388767s) [noise=true , address=45.91.116.6:4984 , piece_id=GQV5UTDPDAYNRXXQH7FL427GSQ5W5BLLOVQIN7NUIQIIIV7AJ4RQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284193911 (302h54m7.776626359s) [noise=true , address=145.79.227.13:4815 , piece_id=GBLGYGVE3WCY27XULJK6UYLUUZFPID33MA7X6N62TWHAMVRH3JBQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284732917 (302h54m7.777165362s) [noise=true , address=77.90.31.11:4556 , piece_id=QCYMCO46JRT2K6H46FYCDTZH6RTTL2Q46HOC2VW44IPTZ2W2KQRQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284833471 (302h54m7.77726592s) [noise=true , address=145.79.227.33:4884 , piece_id=ZTHSKPCM7ZF5BKF26VRUXLJ2N3KGZYFK6HJTKOHYDUJFFYRR7HDA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.285523649 (302h54m7.777956091s) [noise=true , address=145.79.227.32:4854 , piece_id=2YLGSMO4KJ4OTLCQLTFWB6SFEEUPT3X5XKWE7KDQSX7W5LO7FO5Q, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.285585257 (302h54m7.778017699s) [noise=true , address=145.79.227.12:4770 , piece_id=NYBMVQLHBRCGAK2QQHSZ2BRZN3XAMMJKCRYKS4QWI7ZFH64XEGEA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.286297044 (302h54m7.778729499s) [noise=true , address=145.79.227.30:4771 , piece_id=SKXNQMX4RW3KCWKY5TMDPUI5QZJXYROJBMV5Y4P7MAUZUGCSUIEA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.286526017 (302h54m7.778958471s) [noise=true , address=145.79.227.5:4240 , piece_id=Q4PEISD7R7IO4BYBWLADEXE6JBUGQ5AY4J3IPX4P3Q6HZBUSYN7Q, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287104751 (302h54m7.779537193s) [noise=true , address=45.91.116.6:4008 , piece_id=PWJH5FFOUVPV64I4FLTGTBCMYLMIYBEZBZBUCI2WJQBKOPR4KABA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287470770 (302h54m7.779903218s) [noise=true , address=145.79.227.17:4984 , piece_id=ARCUIGNVITSNKFJYAJQEP24HJVBBBDOZ3FRLUQASEZFN2IXGMPLQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287613969 (302h54m7.780046423s) [noise=true , address=145.79.227.14:4843 , piece_id=IXNCU3WCMDYAVT6IE4ETEMTKQROPHV4V77VO5TNNZP5ASMW3WJQQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287960566 (302h54m7.780393019s) [noise=true , address=77.90.31.17:4795 , piece_id=2GUT5J77OOFK3OU6DG5IVXAXCBSV7SZRCXX3TPYZIRMCKGX3JUOQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289210694 (302h54m7.781643146s) [noise=true , address=145.79.227.11:4733 , piece_id=FPN5NMMTUYQHWRQOOYB5RPDKTWHEEWA3CX2OMH233YHTID6MWIFQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289586052 (302h54m7.782018502s) [noise=true , address=145.79.227.37:4589 , piece_id=K4HCLE6PM54ZXJVN5KO626R5UUP32OVEN6DDYIFUM6EYQCRVT7YQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289971344 (302h54m7.782403791s) [noise=true , address=145.79.227.36:4524 , piece_id=RSUO6BNALYS7M6W4RJ55UOD35LNCPD26PQDZBCNY6HHVDFQTRVRQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.849451002 (302h54m8.341883437s) [upload=266387]: Next returned nil segment
Sep 11 05:01:47.850308233 (302h54m8.342740675s) [upload=266387]: Waiting for error group managing segments...
Sep 11 05:01:47.994127497 (302h54m8.486559926s) [noise=true , address=145.79.227.14:4843 , piece_id=IXNCU3WCMDYAVT6IE4ETEMTKQROPHV4V77VO5TNNZP5ASMW3WJQQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.995677280 (302h54m8.488109717s) [noise=true , address=145.79.227.11:4733 , piece_id=FPN5NMMTUYQHWRQOOYB5RPDKTWHEEWA3CX2OMH233YHTID6MWIFQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.996836577 (302h54m8.489269006s) [noise=true , address=145.79.227.30:4771 , piece_id=SKXNQMX4RW3KCWKY5TMDPUI5QZJXYROJBMV5Y4P7MAUZUGCSUIEA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.997314105 (302h54m8.489746535s) [noise=true , address=145.79.227.13:4815 , piece_id=GBLGYGVE3WCY27XULJK6UYLUUZFPID33MA7X6N62TWHAMVRH3JBQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.997716103 (302h54m8.490148547s) [noise=true , address=145.79.227.36:4524 , piece_id=RSUO6BNALYS7M6W4RJ55UOD35LNCPD26PQDZBCNY6HHVDFQTRVRQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.998369140 (302h54m8.490801569s) [noise=true , address=145.79.227.17:4984 , piece_id=ARCUIGNVITSNKFJYAJQEP24HJVBBBDOZ3FRLUQASEZFN2IXGMPLQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.998606161 (302h54m8.491038597s) [noise=true , address=77.90.31.11:4556 , piece_id=QCYMCO46JRT2K6H46FYCDTZH6RTTL2Q46HOC2VW44IPTZ2W2KQRQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.002273466 (302h54m8.494705902s) [noise=true , address=77.90.31.17:4795 , piece_id=2GUT5J77OOFK3OU6DG5IVXAXCBSV7SZRCXX3TPYZIRMCKGX3JUOQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.006810730 (302h54m8.499243159s) [noise=true , address=145.79.227.12:4770 , piece_id=NYBMVQLHBRCGAK2QQHSZ2BRZN3XAMMJKCRYKS4QWI7ZFH64XEGEA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.011564887 (302h54m8.503997331s) [noise=true , address=145.79.227.33:4884 , piece_id=ZTHSKPCM7ZF5BKF26VRUXLJ2N3KGZYFK6HJTKOHYDUJFFYRR7HDA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.023440869 (302h54m8.515873298s) [noise=true , address=45.91.116.11:4147 , piece_id=BPNZTKMMEXM5B6UQSOLWZTZVVT5UOV2QHPSZL742DHQ3AQ3O6ESA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.085368080 (302h54m8.577800508s) [noise=true , address=145.79.227.37:4589 , piece_id=K4HCLE6PM54ZXJVN5KO626R5UUP32OVEN6DDYIFUM6EYQCRVT7YQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:02:46.754779889 (302h55m7.247212318s) [noise=true , address=145.79.227.5:4240 , piece_id=Q4PEISD7R7IO4BYBWLADEXE6JBUGQ5AY4J3IPX4P3Q6HZBUSYN7Q, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:13SaC5StbguPT3abAT9oTX3mZAyGhJcpAA6b7ueyiDFHa8CCEo, address:145.79.227.5:4240): protocol: rpc: dial tcp 145.79.227.5:4240: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.754831971 (302h55m7.247264401s) [noise=true , address=145.79.227.2:4121 , piece_id=KUVJMQVN5VOZCGW5W4DY5BMLI5IH23GSC2IFN6PH6XEA4OT3VUMA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:1Fq1JNvrZLTFTQLVF8jzyavJ6dU1xX96FTGcyxfaHiPdCLu4pB, address:145.79.227.2:4121): protocol: context deadline exceeded; context deadline exceeded
Sep 11 05:02:46.754895261 (302h55m7.247327705s) [noise=true , address=45.91.116.8:4051 , piece_id=PPCINK7K25XXRN3JQPMWVBQVPXXP7UPVSILUGIKWO3OI7EEI42FQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:1preT33ei4yVXXYVLHhaUbHbiyL14oHFEiof6cCWPig83W6CT3, address:45.91.116.8:4051): protocol: write tcp 139.59.144.22:38826->45.91.116.8:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.754988069 (302h55m7.247420503s) [noise=true , address=45.91.116.6:4008 , piece_id=PWJH5FFOUVPV64I4FLTGTBCMYLMIYBEZBZBUCI2WJQBKOPR4KABA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:16KXs6ZXZyroRJibAYU5bsga1RFSpmf7yEtaNG7tamaX7QKvng, address:45.91.116.6:4008): protocol: context deadline exceeded; context deadline exceeded
Sep 11 05:02:46.755109851 (302h55m7.24754228s) [noise=true , address=45.91.116.6:4984 , piece_id=GQV5UTDPDAYNRXXQH7FL427GSQ5W5BLLOVQIN7NUIQIIIV7AJ4RQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt, address:45.91.116.6:4984): protocol: rpc: dial tcp 45.91.116.6:4984: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.755262600 (302h55m7.247695031s) [noise=true , address=145.79.227.32:4854 , piece_id=2YLGSMO4KJ4OTLCQLTFWB6SFEEUPT3X5XKWE7KDQSX7W5LO7FO5Q, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:12eitcwvcNX2GyMiv4GdfDD8wmgb6a1d3ggaGVxbpyKAjPjshmV, address:145.79.227.32:4854): protocol: rpc: dial tcp 145.79.227.32:4854: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.755312278 (302h55m7.247744733s) [seg_pos={0 0}, upload=266387]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Sep 11 05:02:46.866353400 (302h55m7.358785843s) [upload=266387]: Done uploading segments...
Sep 11 05:02:46.866397083 (302h55m7.358829529s) [upload=266387]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 2.396720978s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
913.804549ms
FAIL (2025-09-11 05:43:51 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 913.832926ms
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
2 out of 60 is failed (threshold=3) [....XX......................................................] duration:
337.263312ms target:
https://version.storj.io/
FAIL (2025-09-11 05:43:51 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 337.290998ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..XX........................................................] duration:
2.402394935s
FAIL (2025-09-11 05:43:50 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 2.402422664s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.409810648s
FAIL (2025-09-11 05:43:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
6387]: Got next segment. Inline: false
Sep 11 05:01:47.070094760 (302h54m7.562527201s) [upload=266387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 11 05:01:47.070100812 (302h54m7.562533255s) [upload=266387]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 11 05:01:47.282517324 (302h54m7.774949756s) [seg_pos={0 0}, upload=266387]: Begin upload segment...
Sep 11 05:01:47.282586917 (302h54m7.775019351s) [seg_pos={0 0}, upload=266387]: Done begin upload segment.
Sep 11 05:01:47.282630874 (302h54m7.775063319s) [noise=true , address=145.79.227.2:4121 , piece_id=KUVJMQVN5VOZCGW5W4DY5BMLI5IH23GSC2IFN6PH6XEA4OT3VUMA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283078686 (302h54m7.775511133s) [noise=true , address=45.91.116.8:4051 , piece_id=PPCINK7K25XXRN3JQPMWVBQVPXXP7UPVSILUGIKWO3OI7EEI42FQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283596723 (302h54m7.776029168s) [noise=true , address=45.91.116.11:4147 , piece_id=BPNZTKMMEXM5B6UQSOLWZTZVVT5UOV2QHPSZL742DHQ3AQ3O6ESA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283956314 (302h54m7.776388767s) [noise=true , address=45.91.116.6:4984 , piece_id=GQV5UTDPDAYNRXXQH7FL427GSQ5W5BLLOVQIN7NUIQIIIV7AJ4RQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284193911 (302h54m7.776626359s) [noise=true , address=145.79.227.13:4815 , piece_id=GBLGYGVE3WCY27XULJK6UYLUUZFPID33MA7X6N62TWHAMVRH3JBQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284732917 (302h54m7.777165362s) [noise=true , address=77.90.31.11:4556 , piece_id=QCYMCO46JRT2K6H46FYCDTZH6RTTL2Q46HOC2VW44IPTZ2W2KQRQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284833471 (302h54m7.77726592s) [noise=true , address=145.79.227.33:4884 , piece_id=ZTHSKPCM7ZF5BKF26VRUXLJ2N3KGZYFK6HJTKOHYDUJFFYRR7HDA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.285523649 (302h54m7.777956091s) [noise=true , address=145.79.227.32:4854 , piece_id=2YLGSMO4KJ4OTLCQLTFWB6SFEEUPT3X5XKWE7KDQSX7W5LO7FO5Q, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.285585257 (302h54m7.778017699s) [noise=true , address=145.79.227.12:4770 , piece_id=NYBMVQLHBRCGAK2QQHSZ2BRZN3XAMMJKCRYKS4QWI7ZFH64XEGEA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.286297044 (302h54m7.778729499s) [noise=true , address=145.79.227.30:4771 , piece_id=SKXNQMX4RW3KCWKY5TMDPUI5QZJXYROJBMV5Y4P7MAUZUGCSUIEA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.286526017 (302h54m7.778958471s) [noise=true , address=145.79.227.5:4240 , piece_id=Q4PEISD7R7IO4BYBWLADEXE6JBUGQ5AY4J3IPX4P3Q6HZBUSYN7Q, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287104751 (302h54m7.779537193s) [noise=true , address=45.91.116.6:4008 , piece_id=PWJH5FFOUVPV64I4FLTGTBCMYLMIYBEZBZBUCI2WJQBKOPR4KABA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287470770 (302h54m7.779903218s) [noise=true , address=145.79.227.17:4984 , piece_id=ARCUIGNVITSNKFJYAJQEP24HJVBBBDOZ3FRLUQASEZFN2IXGMPLQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287613969 (302h54m7.780046423s) [noise=true , address=145.79.227.14:4843 , piece_id=IXNCU3WCMDYAVT6IE4ETEMTKQROPHV4V77VO5TNNZP5ASMW3WJQQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287960566 (302h54m7.780393019s) [noise=true , address=77.90.31.17:4795 , piece_id=2GUT5J77OOFK3OU6DG5IVXAXCBSV7SZRCXX3TPYZIRMCKGX3JUOQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289210694 (302h54m7.781643146s) [noise=true , address=145.79.227.11:4733 , piece_id=FPN5NMMTUYQHWRQOOYB5RPDKTWHEEWA3CX2OMH233YHTID6MWIFQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289586052 (302h54m7.782018502s) [noise=true , address=145.79.227.37:4589 , piece_id=K4HCLE6PM54ZXJVN5KO626R5UUP32OVEN6DDYIFUM6EYQCRVT7YQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289971344 (302h54m7.782403791s) [noise=true , address=145.79.227.36:4524 , piece_id=RSUO6BNALYS7M6W4RJ55UOD35LNCPD26PQDZBCNY6HHVDFQTRVRQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.849451002 (302h54m8.341883437s) [upload=266387]: Next returned nil segment
Sep 11 05:01:47.850308233 (302h54m8.342740675s) [upload=266387]: Waiting for error group managing segments...
Sep 11 05:01:47.994127497 (302h54m8.486559926s) [noise=true , address=145.79.227.14:4843 , piece_id=IXNCU3WCMDYAVT6IE4ETEMTKQROPHV4V77VO5TNNZP5ASMW3WJQQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.995677280 (302h54m8.488109717s) [noise=true , address=145.79.227.11:4733 , piece_id=FPN5NMMTUYQHWRQOOYB5RPDKTWHEEWA3CX2OMH233YHTID6MWIFQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.996836577 (302h54m8.489269006s) [noise=true , address=145.79.227.30:4771 , piece_id=SKXNQMX4RW3KCWKY5TMDPUI5QZJXYROJBMV5Y4P7MAUZUGCSUIEA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.997314105 (302h54m8.489746535s) [noise=true , address=145.79.227.13:4815 , piece_id=GBLGYGVE3WCY27XULJK6UYLUUZFPID33MA7X6N62TWHAMVRH3JBQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.997716103 (302h54m8.490148547s) [noise=true , address=145.79.227.36:4524 , piece_id=RSUO6BNALYS7M6W4RJ55UOD35LNCPD26PQDZBCNY6HHVDFQTRVRQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.998369140 (302h54m8.490801569s) [noise=true , address=145.79.227.17:4984 , piece_id=ARCUIGNVITSNKFJYAJQEP24HJVBBBDOZ3FRLUQASEZFN2IXGMPLQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.998606161 (302h54m8.491038597s) [noise=true , address=77.90.31.11:4556 , piece_id=QCYMCO46JRT2K6H46FYCDTZH6RTTL2Q46HOC2VW44IPTZ2W2KQRQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.002273466 (302h54m8.494705902s) [noise=true , address=77.90.31.17:4795 , piece_id=2GUT5J77OOFK3OU6DG5IVXAXCBSV7SZRCXX3TPYZIRMCKGX3JUOQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.006810730 (302h54m8.499243159s) [noise=true , address=145.79.227.12:4770 , piece_id=NYBMVQLHBRCGAK2QQHSZ2BRZN3XAMMJKCRYKS4QWI7ZFH64XEGEA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.011564887 (302h54m8.503997331s) [noise=true , address=145.79.227.33:4884 , piece_id=ZTHSKPCM7ZF5BKF26VRUXLJ2N3KGZYFK6HJTKOHYDUJFFYRR7HDA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.023440869 (302h54m8.515873298s) [noise=true , address=45.91.116.11:4147 , piece_id=BPNZTKMMEXM5B6UQSOLWZTZVVT5UOV2QHPSZL742DHQ3AQ3O6ESA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.085368080 (302h54m8.577800508s) [noise=true , address=145.79.227.37:4589 , piece_id=K4HCLE6PM54ZXJVN5KO626R5UUP32OVEN6DDYIFUM6EYQCRVT7YQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:02:46.754779889 (302h55m7.247212318s) [noise=true , address=145.79.227.5:4240 , piece_id=Q4PEISD7R7IO4BYBWLADEXE6JBUGQ5AY4J3IPX4P3Q6HZBUSYN7Q, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:13SaC5StbguPT3abAT9oTX3mZAyGhJcpAA6b7ueyiDFHa8CCEo, address:145.79.227.5:4240): protocol: rpc: dial tcp 145.79.227.5:4240: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.754831971 (302h55m7.247264401s) [noise=true , address=145.79.227.2:4121 , piece_id=KUVJMQVN5VOZCGW5W4DY5BMLI5IH23GSC2IFN6PH6XEA4OT3VUMA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:1Fq1JNvrZLTFTQLVF8jzyavJ6dU1xX96FTGcyxfaHiPdCLu4pB, address:145.79.227.2:4121): protocol: context deadline exceeded; context deadline exceeded
Sep 11 05:02:46.754895261 (302h55m7.247327705s) [noise=true , address=45.91.116.8:4051 , piece_id=PPCINK7K25XXRN3JQPMWVBQVPXXP7UPVSILUGIKWO3OI7EEI42FQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:1preT33ei4yVXXYVLHhaUbHbiyL14oHFEiof6cCWPig83W6CT3, address:45.91.116.8:4051): protocol: write tcp 139.59.144.22:38826->45.91.116.8:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.754988069 (302h55m7.247420503s) [noise=true , address=45.91.116.6:4008 , piece_id=PWJH5FFOUVPV64I4FLTGTBCMYLMIYBEZBZBUCI2WJQBKOPR4KABA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:16KXs6ZXZyroRJibAYU5bsga1RFSpmf7yEtaNG7tamaX7QKvng, address:45.91.116.6:4008): protocol: context deadline exceeded; context deadline exceeded
Sep 11 05:02:46.755109851 (302h55m7.24754228s) [noise=true , address=45.91.116.6:4984 , piece_id=GQV5UTDPDAYNRXXQH7FL427GSQ5W5BLLOVQIN7NUIQIIIV7AJ4RQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt, address:45.91.116.6:4984): protocol: rpc: dial tcp 45.91.116.6:4984: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.755262600 (302h55m7.247695031s) [noise=true , address=145.79.227.32:4854 , piece_id=2YLGSMO4KJ4OTLCQLTFWB6SFEEUPT3X5XKWE7KDQSX7W5LO7FO5Q, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:12eitcwvcNX2GyMiv4GdfDD8wmgb6a1d3ggaGVxbpyKAjPjshmV, address:145.79.227.32:4854): protocol: rpc: dial tcp 145.79.227.32:4854: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.755312278 (302h55m7.247744733s) [seg_pos={0 0}, upload=266387]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Sep 11 05:02:46.866353400 (302h55m7.358785843s) [upload=266387]: Done uploading segments...
Sep 11 05:02:46.866397083 (302h55m7.358829529s) [upload=266387]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 2.409846102s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
509.441888ms target:
https://ap1.storj.io/login
FAIL (2025-09-11 05:43:27 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout
Duration: 509.469445ms
core
satellite
ap1
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration:
334.829154ms target:
https://version.storj.io/
FAIL (2025-09-11 05:43:12 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 334.859897ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
1.113919027s
FAIL (2025-09-11 05:43:08 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.113953862s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...XX.......................................................] duration:
2.527090722s
FAIL (2025-09-11 05:42:51 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 2.527130145s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
2.391422761s
FAIL (2025-09-11 05:42:45 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
6387]: Got next segment. Inline: false
Sep 11 05:01:47.070094760 (302h54m7.562527201s) [upload=266387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 11 05:01:47.070100812 (302h54m7.562533255s) [upload=266387]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 11 05:01:47.282517324 (302h54m7.774949756s) [seg_pos={0 0}, upload=266387]: Begin upload segment...
Sep 11 05:01:47.282586917 (302h54m7.775019351s) [seg_pos={0 0}, upload=266387]: Done begin upload segment.
Sep 11 05:01:47.282630874 (302h54m7.775063319s) [noise=true , address=145.79.227.2:4121 , piece_id=KUVJMQVN5VOZCGW5W4DY5BMLI5IH23GSC2IFN6PH6XEA4OT3VUMA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283078686 (302h54m7.775511133s) [noise=true , address=45.91.116.8:4051 , piece_id=PPCINK7K25XXRN3JQPMWVBQVPXXP7UPVSILUGIKWO3OI7EEI42FQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283596723 (302h54m7.776029168s) [noise=true , address=45.91.116.11:4147 , piece_id=BPNZTKMMEXM5B6UQSOLWZTZVVT5UOV2QHPSZL742DHQ3AQ3O6ESA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283956314 (302h54m7.776388767s) [noise=true , address=45.91.116.6:4984 , piece_id=GQV5UTDPDAYNRXXQH7FL427GSQ5W5BLLOVQIN7NUIQIIIV7AJ4RQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284193911 (302h54m7.776626359s) [noise=true , address=145.79.227.13:4815 , piece_id=GBLGYGVE3WCY27XULJK6UYLUUZFPID33MA7X6N62TWHAMVRH3JBQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284732917 (302h54m7.777165362s) [noise=true , address=77.90.31.11:4556 , piece_id=QCYMCO46JRT2K6H46FYCDTZH6RTTL2Q46HOC2VW44IPTZ2W2KQRQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284833471 (302h54m7.77726592s) [noise=true , address=145.79.227.33:4884 , piece_id=ZTHSKPCM7ZF5BKF26VRUXLJ2N3KGZYFK6HJTKOHYDUJFFYRR7HDA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.285523649 (302h54m7.777956091s) [noise=true , address=145.79.227.32:4854 , piece_id=2YLGSMO4KJ4OTLCQLTFWB6SFEEUPT3X5XKWE7KDQSX7W5LO7FO5Q, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.285585257 (302h54m7.778017699s) [noise=true , address=145.79.227.12:4770 , piece_id=NYBMVQLHBRCGAK2QQHSZ2BRZN3XAMMJKCRYKS4QWI7ZFH64XEGEA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.286297044 (302h54m7.778729499s) [noise=true , address=145.79.227.30:4771 , piece_id=SKXNQMX4RW3KCWKY5TMDPUI5QZJXYROJBMV5Y4P7MAUZUGCSUIEA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.286526017 (302h54m7.778958471s) [noise=true , address=145.79.227.5:4240 , piece_id=Q4PEISD7R7IO4BYBWLADEXE6JBUGQ5AY4J3IPX4P3Q6HZBUSYN7Q, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287104751 (302h54m7.779537193s) [noise=true , address=45.91.116.6:4008 , piece_id=PWJH5FFOUVPV64I4FLTGTBCMYLMIYBEZBZBUCI2WJQBKOPR4KABA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287470770 (302h54m7.779903218s) [noise=true , address=145.79.227.17:4984 , piece_id=ARCUIGNVITSNKFJYAJQEP24HJVBBBDOZ3FRLUQASEZFN2IXGMPLQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287613969 (302h54m7.780046423s) [noise=true , address=145.79.227.14:4843 , piece_id=IXNCU3WCMDYAVT6IE4ETEMTKQROPHV4V77VO5TNNZP5ASMW3WJQQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287960566 (302h54m7.780393019s) [noise=true , address=77.90.31.17:4795 , piece_id=2GUT5J77OOFK3OU6DG5IVXAXCBSV7SZRCXX3TPYZIRMCKGX3JUOQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289210694 (302h54m7.781643146s) [noise=true , address=145.79.227.11:4733 , piece_id=FPN5NMMTUYQHWRQOOYB5RPDKTWHEEWA3CX2OMH233YHTID6MWIFQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289586052 (302h54m7.782018502s) [noise=true , address=145.79.227.37:4589 , piece_id=K4HCLE6PM54ZXJVN5KO626R5UUP32OVEN6DDYIFUM6EYQCRVT7YQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289971344 (302h54m7.782403791s) [noise=true , address=145.79.227.36:4524 , piece_id=RSUO6BNALYS7M6W4RJ55UOD35LNCPD26PQDZBCNY6HHVDFQTRVRQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.849451002 (302h54m8.341883437s) [upload=266387]: Next returned nil segment
Sep 11 05:01:47.850308233 (302h54m8.342740675s) [upload=266387]: Waiting for error group managing segments...
Sep 11 05:01:47.994127497 (302h54m8.486559926s) [noise=true , address=145.79.227.14:4843 , piece_id=IXNCU3WCMDYAVT6IE4ETEMTKQROPHV4V77VO5TNNZP5ASMW3WJQQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.995677280 (302h54m8.488109717s) [noise=true , address=145.79.227.11:4733 , piece_id=FPN5NMMTUYQHWRQOOYB5RPDKTWHEEWA3CX2OMH233YHTID6MWIFQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.996836577 (302h54m8.489269006s) [noise=true , address=145.79.227.30:4771 , piece_id=SKXNQMX4RW3KCWKY5TMDPUI5QZJXYROJBMV5Y4P7MAUZUGCSUIEA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.997314105 (302h54m8.489746535s) [noise=true , address=145.79.227.13:4815 , piece_id=GBLGYGVE3WCY27XULJK6UYLUUZFPID33MA7X6N62TWHAMVRH3JBQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.997716103 (302h54m8.490148547s) [noise=true , address=145.79.227.36:4524 , piece_id=RSUO6BNALYS7M6W4RJ55UOD35LNCPD26PQDZBCNY6HHVDFQTRVRQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.998369140 (302h54m8.490801569s) [noise=true , address=145.79.227.17:4984 , piece_id=ARCUIGNVITSNKFJYAJQEP24HJVBBBDOZ3FRLUQASEZFN2IXGMPLQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.998606161 (302h54m8.491038597s) [noise=true , address=77.90.31.11:4556 , piece_id=QCYMCO46JRT2K6H46FYCDTZH6RTTL2Q46HOC2VW44IPTZ2W2KQRQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.002273466 (302h54m8.494705902s) [noise=true , address=77.90.31.17:4795 , piece_id=2GUT5J77OOFK3OU6DG5IVXAXCBSV7SZRCXX3TPYZIRMCKGX3JUOQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.006810730 (302h54m8.499243159s) [noise=true , address=145.79.227.12:4770 , piece_id=NYBMVQLHBRCGAK2QQHSZ2BRZN3XAMMJKCRYKS4QWI7ZFH64XEGEA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.011564887 (302h54m8.503997331s) [noise=true , address=145.79.227.33:4884 , piece_id=ZTHSKPCM7ZF5BKF26VRUXLJ2N3KGZYFK6HJTKOHYDUJFFYRR7HDA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.023440869 (302h54m8.515873298s) [noise=true , address=45.91.116.11:4147 , piece_id=BPNZTKMMEXM5B6UQSOLWZTZVVT5UOV2QHPSZL742DHQ3AQ3O6ESA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.085368080 (302h54m8.577800508s) [noise=true , address=145.79.227.37:4589 , piece_id=K4HCLE6PM54ZXJVN5KO626R5UUP32OVEN6DDYIFUM6EYQCRVT7YQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:02:46.754779889 (302h55m7.247212318s) [noise=true , address=145.79.227.5:4240 , piece_id=Q4PEISD7R7IO4BYBWLADEXE6JBUGQ5AY4J3IPX4P3Q6HZBUSYN7Q, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:13SaC5StbguPT3abAT9oTX3mZAyGhJcpAA6b7ueyiDFHa8CCEo, address:145.79.227.5:4240): protocol: rpc: dial tcp 145.79.227.5:4240: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.754831971 (302h55m7.247264401s) [noise=true , address=145.79.227.2:4121 , piece_id=KUVJMQVN5VOZCGW5W4DY5BMLI5IH23GSC2IFN6PH6XEA4OT3VUMA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:1Fq1JNvrZLTFTQLVF8jzyavJ6dU1xX96FTGcyxfaHiPdCLu4pB, address:145.79.227.2:4121): protocol: context deadline exceeded; context deadline exceeded
Sep 11 05:02:46.754895261 (302h55m7.247327705s) [noise=true , address=45.91.116.8:4051 , piece_id=PPCINK7K25XXRN3JQPMWVBQVPXXP7UPVSILUGIKWO3OI7EEI42FQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:1preT33ei4yVXXYVLHhaUbHbiyL14oHFEiof6cCWPig83W6CT3, address:45.91.116.8:4051): protocol: write tcp 139.59.144.22:38826->45.91.116.8:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.754988069 (302h55m7.247420503s) [noise=true , address=45.91.116.6:4008 , piece_id=PWJH5FFOUVPV64I4FLTGTBCMYLMIYBEZBZBUCI2WJQBKOPR4KABA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:16KXs6ZXZyroRJibAYU5bsga1RFSpmf7yEtaNG7tamaX7QKvng, address:45.91.116.6:4008): protocol: context deadline exceeded; context deadline exceeded
Sep 11 05:02:46.755109851 (302h55m7.24754228s) [noise=true , address=45.91.116.6:4984 , piece_id=GQV5UTDPDAYNRXXQH7FL427GSQ5W5BLLOVQIN7NUIQIIIV7AJ4RQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt, address:45.91.116.6:4984): protocol: rpc: dial tcp 45.91.116.6:4984: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.755262600 (302h55m7.247695031s) [noise=true , address=145.79.227.32:4854 , piece_id=2YLGSMO4KJ4OTLCQLTFWB6SFEEUPT3X5XKWE7KDQSX7W5LO7FO5Q, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:12eitcwvcNX2GyMiv4GdfDD8wmgb6a1d3ggaGVxbpyKAjPjshmV, address:145.79.227.32:4854): protocol: rpc: dial tcp 145.79.227.32:4854: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.755312278 (302h55m7.247744733s) [seg_pos={0 0}, upload=266387]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Sep 11 05:02:46.866353400 (302h55m7.358785843s) [upload=266387]: Done uploading segments...
Sep 11 05:02:46.866397083 (302h55m7.358829529s) [upload=266387]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 2.391459598s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
506.434499ms target:
https://ap1.storj.io/login
FAIL (2025-09-11 05:42:40 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout
Duration: 506.463694ms
core
satellite
ap1
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
2 out of 60 is failed (threshold=3) [......XX....................................................] duration:
343.982307ms target:
https://version.storj.io/
FAIL (2025-09-11 05:42:29 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 344.010886ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
1.104519787s
FAIL (2025-09-11 05:42:25 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.104560682s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
506.285071ms target:
https://ap1.storj.io/login
FAIL (2025-09-11 05:42:24 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout
Duration: 506.311622ms
core
satellite
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
2.342947642s
FAIL (2025-09-11 05:42:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
6387]: Got next segment. Inline: false
Sep 11 05:01:47.070094760 (302h54m7.562527201s) [upload=266387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 11 05:01:47.070100812 (302h54m7.562533255s) [upload=266387]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 11 05:01:47.282517324 (302h54m7.774949756s) [seg_pos={0 0}, upload=266387]: Begin upload segment...
Sep 11 05:01:47.282586917 (302h54m7.775019351s) [seg_pos={0 0}, upload=266387]: Done begin upload segment.
Sep 11 05:01:47.282630874 (302h54m7.775063319s) [noise=true , address=145.79.227.2:4121 , piece_id=KUVJMQVN5VOZCGW5W4DY5BMLI5IH23GSC2IFN6PH6XEA4OT3VUMA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283078686 (302h54m7.775511133s) [noise=true , address=45.91.116.8:4051 , piece_id=PPCINK7K25XXRN3JQPMWVBQVPXXP7UPVSILUGIKWO3OI7EEI42FQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283596723 (302h54m7.776029168s) [noise=true , address=45.91.116.11:4147 , piece_id=BPNZTKMMEXM5B6UQSOLWZTZVVT5UOV2QHPSZL742DHQ3AQ3O6ESA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283956314 (302h54m7.776388767s) [noise=true , address=45.91.116.6:4984 , piece_id=GQV5UTDPDAYNRXXQH7FL427GSQ5W5BLLOVQIN7NUIQIIIV7AJ4RQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284193911 (302h54m7.776626359s) [noise=true , address=145.79.227.13:4815 , piece_id=GBLGYGVE3WCY27XULJK6UYLUUZFPID33MA7X6N62TWHAMVRH3JBQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284732917 (302h54m7.777165362s) [noise=true , address=77.90.31.11:4556 , piece_id=QCYMCO46JRT2K6H46FYCDTZH6RTTL2Q46HOC2VW44IPTZ2W2KQRQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284833471 (302h54m7.77726592s) [noise=true , address=145.79.227.33:4884 , piece_id=ZTHSKPCM7ZF5BKF26VRUXLJ2N3KGZYFK6HJTKOHYDUJFFYRR7HDA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.285523649 (302h54m7.777956091s) [noise=true , address=145.79.227.32:4854 , piece_id=2YLGSMO4KJ4OTLCQLTFWB6SFEEUPT3X5XKWE7KDQSX7W5LO7FO5Q, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.285585257 (302h54m7.778017699s) [noise=true , address=145.79.227.12:4770 , piece_id=NYBMVQLHBRCGAK2QQHSZ2BRZN3XAMMJKCRYKS4QWI7ZFH64XEGEA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.286297044 (302h54m7.778729499s) [noise=true , address=145.79.227.30:4771 , piece_id=SKXNQMX4RW3KCWKY5TMDPUI5QZJXYROJBMV5Y4P7MAUZUGCSUIEA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.286526017 (302h54m7.778958471s) [noise=true , address=145.79.227.5:4240 , piece_id=Q4PEISD7R7IO4BYBWLADEXE6JBUGQ5AY4J3IPX4P3Q6HZBUSYN7Q, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287104751 (302h54m7.779537193s) [noise=true , address=45.91.116.6:4008 , piece_id=PWJH5FFOUVPV64I4FLTGTBCMYLMIYBEZBZBUCI2WJQBKOPR4KABA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287470770 (302h54m7.779903218s) [noise=true , address=145.79.227.17:4984 , piece_id=ARCUIGNVITSNKFJYAJQEP24HJVBBBDOZ3FRLUQASEZFN2IXGMPLQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287613969 (302h54m7.780046423s) [noise=true , address=145.79.227.14:4843 , piece_id=IXNCU3WCMDYAVT6IE4ETEMTKQROPHV4V77VO5TNNZP5ASMW3WJQQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287960566 (302h54m7.780393019s) [noise=true , address=77.90.31.17:4795 , piece_id=2GUT5J77OOFK3OU6DG5IVXAXCBSV7SZRCXX3TPYZIRMCKGX3JUOQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289210694 (302h54m7.781643146s) [noise=true , address=145.79.227.11:4733 , piece_id=FPN5NMMTUYQHWRQOOYB5RPDKTWHEEWA3CX2OMH233YHTID6MWIFQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289586052 (302h54m7.782018502s) [noise=true , address=145.79.227.37:4589 , piece_id=K4HCLE6PM54ZXJVN5KO626R5UUP32OVEN6DDYIFUM6EYQCRVT7YQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289971344 (302h54m7.782403791s) [noise=true , address=145.79.227.36:4524 , piece_id=RSUO6BNALYS7M6W4RJ55UOD35LNCPD26PQDZBCNY6HHVDFQTRVRQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.849451002 (302h54m8.341883437s) [upload=266387]: Next returned nil segment
Sep 11 05:01:47.850308233 (302h54m8.342740675s) [upload=266387]: Waiting for error group managing segments...
Sep 11 05:01:47.994127497 (302h54m8.486559926s) [noise=true , address=145.79.227.14:4843 , piece_id=IXNCU3WCMDYAVT6IE4ETEMTKQROPHV4V77VO5TNNZP5ASMW3WJQQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.995677280 (302h54m8.488109717s) [noise=true , address=145.79.227.11:4733 , piece_id=FPN5NMMTUYQHWRQOOYB5RPDKTWHEEWA3CX2OMH233YHTID6MWIFQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.996836577 (302h54m8.489269006s) [noise=true , address=145.79.227.30:4771 , piece_id=SKXNQMX4RW3KCWKY5TMDPUI5QZJXYROJBMV5Y4P7MAUZUGCSUIEA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.997314105 (302h54m8.489746535s) [noise=true , address=145.79.227.13:4815 , piece_id=GBLGYGVE3WCY27XULJK6UYLUUZFPID33MA7X6N62TWHAMVRH3JBQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.997716103 (302h54m8.490148547s) [noise=true , address=145.79.227.36:4524 , piece_id=RSUO6BNALYS7M6W4RJ55UOD35LNCPD26PQDZBCNY6HHVDFQTRVRQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.998369140 (302h54m8.490801569s) [noise=true , address=145.79.227.17:4984 , piece_id=ARCUIGNVITSNKFJYAJQEP24HJVBBBDOZ3FRLUQASEZFN2IXGMPLQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.998606161 (302h54m8.491038597s) [noise=true , address=77.90.31.11:4556 , piece_id=QCYMCO46JRT2K6H46FYCDTZH6RTTL2Q46HOC2VW44IPTZ2W2KQRQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.002273466 (302h54m8.494705902s) [noise=true , address=77.90.31.17:4795 , piece_id=2GUT5J77OOFK3OU6DG5IVXAXCBSV7SZRCXX3TPYZIRMCKGX3JUOQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.006810730 (302h54m8.499243159s) [noise=true , address=145.79.227.12:4770 , piece_id=NYBMVQLHBRCGAK2QQHSZ2BRZN3XAMMJKCRYKS4QWI7ZFH64XEGEA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.011564887 (302h54m8.503997331s) [noise=true , address=145.79.227.33:4884 , piece_id=ZTHSKPCM7ZF5BKF26VRUXLJ2N3KGZYFK6HJTKOHYDUJFFYRR7HDA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.023440869 (302h54m8.515873298s) [noise=true , address=45.91.116.11:4147 , piece_id=BPNZTKMMEXM5B6UQSOLWZTZVVT5UOV2QHPSZL742DHQ3AQ3O6ESA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.085368080 (302h54m8.577800508s) [noise=true , address=145.79.227.37:4589 , piece_id=K4HCLE6PM54ZXJVN5KO626R5UUP32OVEN6DDYIFUM6EYQCRVT7YQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:02:46.754779889 (302h55m7.247212318s) [noise=true , address=145.79.227.5:4240 , piece_id=Q4PEISD7R7IO4BYBWLADEXE6JBUGQ5AY4J3IPX4P3Q6HZBUSYN7Q, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:13SaC5StbguPT3abAT9oTX3mZAyGhJcpAA6b7ueyiDFHa8CCEo, address:145.79.227.5:4240): protocol: rpc: dial tcp 145.79.227.5:4240: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.754831971 (302h55m7.247264401s) [noise=true , address=145.79.227.2:4121 , piece_id=KUVJMQVN5VOZCGW5W4DY5BMLI5IH23GSC2IFN6PH6XEA4OT3VUMA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:1Fq1JNvrZLTFTQLVF8jzyavJ6dU1xX96FTGcyxfaHiPdCLu4pB, address:145.79.227.2:4121): protocol: context deadline exceeded; context deadline exceeded
Sep 11 05:02:46.754895261 (302h55m7.247327705s) [noise=true , address=45.91.116.8:4051 , piece_id=PPCINK7K25XXRN3JQPMWVBQVPXXP7UPVSILUGIKWO3OI7EEI42FQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:1preT33ei4yVXXYVLHhaUbHbiyL14oHFEiof6cCWPig83W6CT3, address:45.91.116.8:4051): protocol: write tcp 139.59.144.22:38826->45.91.116.8:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.754988069 (302h55m7.247420503s) [noise=true , address=45.91.116.6:4008 , piece_id=PWJH5FFOUVPV64I4FLTGTBCMYLMIYBEZBZBUCI2WJQBKOPR4KABA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:16KXs6ZXZyroRJibAYU5bsga1RFSpmf7yEtaNG7tamaX7QKvng, address:45.91.116.6:4008): protocol: context deadline exceeded; context deadline exceeded
Sep 11 05:02:46.755109851 (302h55m7.24754228s) [noise=true , address=45.91.116.6:4984 , piece_id=GQV5UTDPDAYNRXXQH7FL427GSQ5W5BLLOVQIN7NUIQIIIV7AJ4RQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt, address:45.91.116.6:4984): protocol: rpc: dial tcp 45.91.116.6:4984: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.755262600 (302h55m7.247695031s) [noise=true , address=145.79.227.32:4854 , piece_id=2YLGSMO4KJ4OTLCQLTFWB6SFEEUPT3X5XKWE7KDQSX7W5LO7FO5Q, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:12eitcwvcNX2GyMiv4GdfDD8wmgb6a1d3ggaGVxbpyKAjPjshmV, address:145.79.227.32:4854): protocol: rpc: dial tcp 145.79.227.32:4854: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.755312278 (302h55m7.247744733s) [seg_pos={0 0}, upload=266387]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Sep 11 05:02:46.866353400 (302h55m7.358785843s) [upload=266387]: Done uploading segments...
Sep 11 05:02:46.866397083 (302h55m7.358829529s) [upload=266387]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 2.342983019s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.382733797s
FAIL (2025-09-11 05:41:52 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
6387]: Got next segment. Inline: false
Sep 11 05:01:47.070094760 (302h54m7.562527201s) [upload=266387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 11 05:01:47.070100812 (302h54m7.562533255s) [upload=266387]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 11 05:01:47.282517324 (302h54m7.774949756s) [seg_pos={0 0}, upload=266387]: Begin upload segment...
Sep 11 05:01:47.282586917 (302h54m7.775019351s) [seg_pos={0 0}, upload=266387]: Done begin upload segment.
Sep 11 05:01:47.282630874 (302h54m7.775063319s) [noise=true , address=145.79.227.2:4121 , piece_id=KUVJMQVN5VOZCGW5W4DY5BMLI5IH23GSC2IFN6PH6XEA4OT3VUMA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283078686 (302h54m7.775511133s) [noise=true , address=45.91.116.8:4051 , piece_id=PPCINK7K25XXRN3JQPMWVBQVPXXP7UPVSILUGIKWO3OI7EEI42FQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283596723 (302h54m7.776029168s) [noise=true , address=45.91.116.11:4147 , piece_id=BPNZTKMMEXM5B6UQSOLWZTZVVT5UOV2QHPSZL742DHQ3AQ3O6ESA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283956314 (302h54m7.776388767s) [noise=true , address=45.91.116.6:4984 , piece_id=GQV5UTDPDAYNRXXQH7FL427GSQ5W5BLLOVQIN7NUIQIIIV7AJ4RQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284193911 (302h54m7.776626359s) [noise=true , address=145.79.227.13:4815 , piece_id=GBLGYGVE3WCY27XULJK6UYLUUZFPID33MA7X6N62TWHAMVRH3JBQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284732917 (302h54m7.777165362s) [noise=true , address=77.90.31.11:4556 , piece_id=QCYMCO46JRT2K6H46FYCDTZH6RTTL2Q46HOC2VW44IPTZ2W2KQRQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284833471 (302h54m7.77726592s) [noise=true , address=145.79.227.33:4884 , piece_id=ZTHSKPCM7ZF5BKF26VRUXLJ2N3KGZYFK6HJTKOHYDUJFFYRR7HDA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.285523649 (302h54m7.777956091s) [noise=true , address=145.79.227.32:4854 , piece_id=2YLGSMO4KJ4OTLCQLTFWB6SFEEUPT3X5XKWE7KDQSX7W5LO7FO5Q, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.285585257 (302h54m7.778017699s) [noise=true , address=145.79.227.12:4770 , piece_id=NYBMVQLHBRCGAK2QQHSZ2BRZN3XAMMJKCRYKS4QWI7ZFH64XEGEA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.286297044 (302h54m7.778729499s) [noise=true , address=145.79.227.30:4771 , piece_id=SKXNQMX4RW3KCWKY5TMDPUI5QZJXYROJBMV5Y4P7MAUZUGCSUIEA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.286526017 (302h54m7.778958471s) [noise=true , address=145.79.227.5:4240 , piece_id=Q4PEISD7R7IO4BYBWLADEXE6JBUGQ5AY4J3IPX4P3Q6HZBUSYN7Q, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287104751 (302h54m7.779537193s) [noise=true , address=45.91.116.6:4008 , piece_id=PWJH5FFOUVPV64I4FLTGTBCMYLMIYBEZBZBUCI2WJQBKOPR4KABA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287470770 (302h54m7.779903218s) [noise=true , address=145.79.227.17:4984 , piece_id=ARCUIGNVITSNKFJYAJQEP24HJVBBBDOZ3FRLUQASEZFN2IXGMPLQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287613969 (302h54m7.780046423s) [noise=true , address=145.79.227.14:4843 , piece_id=IXNCU3WCMDYAVT6IE4ETEMTKQROPHV4V77VO5TNNZP5ASMW3WJQQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287960566 (302h54m7.780393019s) [noise=true , address=77.90.31.17:4795 , piece_id=2GUT5J77OOFK3OU6DG5IVXAXCBSV7SZRCXX3TPYZIRMCKGX3JUOQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289210694 (302h54m7.781643146s) [noise=true , address=145.79.227.11:4733 , piece_id=FPN5NMMTUYQHWRQOOYB5RPDKTWHEEWA3CX2OMH233YHTID6MWIFQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289586052 (302h54m7.782018502s) [noise=true , address=145.79.227.37:4589 , piece_id=K4HCLE6PM54ZXJVN5KO626R5UUP32OVEN6DDYIFUM6EYQCRVT7YQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289971344 (302h54m7.782403791s) [noise=true , address=145.79.227.36:4524 , piece_id=RSUO6BNALYS7M6W4RJ55UOD35LNCPD26PQDZBCNY6HHVDFQTRVRQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.849451002 (302h54m8.341883437s) [upload=266387]: Next returned nil segment
Sep 11 05:01:47.850308233 (302h54m8.342740675s) [upload=266387]: Waiting for error group managing segments...
Sep 11 05:01:47.994127497 (302h54m8.486559926s) [noise=true , address=145.79.227.14:4843 , piece_id=IXNCU3WCMDYAVT6IE4ETEMTKQROPHV4V77VO5TNNZP5ASMW3WJQQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.995677280 (302h54m8.488109717s) [noise=true , address=145.79.227.11:4733 , piece_id=FPN5NMMTUYQHWRQOOYB5RPDKTWHEEWA3CX2OMH233YHTID6MWIFQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.996836577 (302h54m8.489269006s) [noise=true , address=145.79.227.30:4771 , piece_id=SKXNQMX4RW3KCWKY5TMDPUI5QZJXYROJBMV5Y4P7MAUZUGCSUIEA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.997314105 (302h54m8.489746535s) [noise=true , address=145.79.227.13:4815 , piece_id=GBLGYGVE3WCY27XULJK6UYLUUZFPID33MA7X6N62TWHAMVRH3JBQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.997716103 (302h54m8.490148547s) [noise=true , address=145.79.227.36:4524 , piece_id=RSUO6BNALYS7M6W4RJ55UOD35LNCPD26PQDZBCNY6HHVDFQTRVRQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.998369140 (302h54m8.490801569s) [noise=true , address=145.79.227.17:4984 , piece_id=ARCUIGNVITSNKFJYAJQEP24HJVBBBDOZ3FRLUQASEZFN2IXGMPLQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.998606161 (302h54m8.491038597s) [noise=true , address=77.90.31.11:4556 , piece_id=QCYMCO46JRT2K6H46FYCDTZH6RTTL2Q46HOC2VW44IPTZ2W2KQRQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.002273466 (302h54m8.494705902s) [noise=true , address=77.90.31.17:4795 , piece_id=2GUT5J77OOFK3OU6DG5IVXAXCBSV7SZRCXX3TPYZIRMCKGX3JUOQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.006810730 (302h54m8.499243159s) [noise=true , address=145.79.227.12:4770 , piece_id=NYBMVQLHBRCGAK2QQHSZ2BRZN3XAMMJKCRYKS4QWI7ZFH64XEGEA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.011564887 (302h54m8.503997331s) [noise=true , address=145.79.227.33:4884 , piece_id=ZTHSKPCM7ZF5BKF26VRUXLJ2N3KGZYFK6HJTKOHYDUJFFYRR7HDA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.023440869 (302h54m8.515873298s) [noise=true , address=45.91.116.11:4147 , piece_id=BPNZTKMMEXM5B6UQSOLWZTZVVT5UOV2QHPSZL742DHQ3AQ3O6ESA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.085368080 (302h54m8.577800508s) [noise=true , address=145.79.227.37:4589 , piece_id=K4HCLE6PM54ZXJVN5KO626R5UUP32OVEN6DDYIFUM6EYQCRVT7YQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:02:46.754779889 (302h55m7.247212318s) [noise=true , address=145.79.227.5:4240 , piece_id=Q4PEISD7R7IO4BYBWLADEXE6JBUGQ5AY4J3IPX4P3Q6HZBUSYN7Q, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:13SaC5StbguPT3abAT9oTX3mZAyGhJcpAA6b7ueyiDFHa8CCEo, address:145.79.227.5:4240): protocol: rpc: dial tcp 145.79.227.5:4240: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.754831971 (302h55m7.247264401s) [noise=true , address=145.79.227.2:4121 , piece_id=KUVJMQVN5VOZCGW5W4DY5BMLI5IH23GSC2IFN6PH6XEA4OT3VUMA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:1Fq1JNvrZLTFTQLVF8jzyavJ6dU1xX96FTGcyxfaHiPdCLu4pB, address:145.79.227.2:4121): protocol: context deadline exceeded; context deadline exceeded
Sep 11 05:02:46.754895261 (302h55m7.247327705s) [noise=true , address=45.91.116.8:4051 , piece_id=PPCINK7K25XXRN3JQPMWVBQVPXXP7UPVSILUGIKWO3OI7EEI42FQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:1preT33ei4yVXXYVLHhaUbHbiyL14oHFEiof6cCWPig83W6CT3, address:45.91.116.8:4051): protocol: write tcp 139.59.144.22:38826->45.91.116.8:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.754988069 (302h55m7.247420503s) [noise=true , address=45.91.116.6:4008 , piece_id=PWJH5FFOUVPV64I4FLTGTBCMYLMIYBEZBZBUCI2WJQBKOPR4KABA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:16KXs6ZXZyroRJibAYU5bsga1RFSpmf7yEtaNG7tamaX7QKvng, address:45.91.116.6:4008): protocol: context deadline exceeded; context deadline exceeded
Sep 11 05:02:46.755109851 (302h55m7.24754228s) [noise=true , address=45.91.116.6:4984 , piece_id=GQV5UTDPDAYNRXXQH7FL427GSQ5W5BLLOVQIN7NUIQIIIV7AJ4RQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt, address:45.91.116.6:4984): protocol: rpc: dial tcp 45.91.116.6:4984: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.755262600 (302h55m7.247695031s) [noise=true , address=145.79.227.32:4854 , piece_id=2YLGSMO4KJ4OTLCQLTFWB6SFEEUPT3X5XKWE7KDQSX7W5LO7FO5Q, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:12eitcwvcNX2GyMiv4GdfDD8wmgb6a1d3ggaGVxbpyKAjPjshmV, address:145.79.227.32:4854): protocol: rpc: dial tcp 145.79.227.32:4854: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.755312278 (302h55m7.247744733s) [seg_pos={0 0}, upload=266387]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Sep 11 05:02:46.866353400 (302h55m7.358785843s) [upload=266387]: Done uploading segments...
Sep 11 05:02:46.866397083 (302h55m7.358829529s) [upload=266387]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 2.38276444s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
525.548901ms target:
https://ap1.storj.io/login
FAIL (2025-09-11 05:41:45 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout
Duration: 525.573905ms
core
satellite
ap1
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
2 out of 60 is failed (threshold=3) [.......XX...................................................] duration:
338.552048ms target:
https://version.storj.io/
FAIL (2025-09-11 05:41:44 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 338.586252ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....XX......................................................] duration:
2.380518008s
FAIL (2025-09-11 05:41:41 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 2.380548386s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
2 out of 60 is failed (threshold=3) [........XX..................................................] duration:
338.126447ms target:
https://version.storj.io/
FAIL (2025-09-11 05:41:22 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 34.173.164.90:443: i/o timeout
Duration: 338.219894ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
889.819475ms
FAIL (2025-09-11 05:41:16 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 889.849045ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
1.007085969s
FAIL (2025-09-11 05:40:56 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.007123216s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
524.8616ms target:
https://ap1.storj.io/login
FAIL (2025-09-11 05:40:53 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout
Duration: 524.902435ms
core
satellite
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
2.422011007s
FAIL (2025-09-11 05:40:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
6387]: Got next segment. Inline: false
Sep 11 05:01:47.070094760 (302h54m7.562527201s) [upload=266387]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 11 05:01:47.070100812 (302h54m7.562533255s) [upload=266387]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 11 05:01:47.282517324 (302h54m7.774949756s) [seg_pos={0 0}, upload=266387]: Begin upload segment...
Sep 11 05:01:47.282586917 (302h54m7.775019351s) [seg_pos={0 0}, upload=266387]: Done begin upload segment.
Sep 11 05:01:47.282630874 (302h54m7.775063319s) [noise=true , address=145.79.227.2:4121 , piece_id=KUVJMQVN5VOZCGW5W4DY5BMLI5IH23GSC2IFN6PH6XEA4OT3VUMA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283078686 (302h54m7.775511133s) [noise=true , address=45.91.116.8:4051 , piece_id=PPCINK7K25XXRN3JQPMWVBQVPXXP7UPVSILUGIKWO3OI7EEI42FQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283596723 (302h54m7.776029168s) [noise=true , address=45.91.116.11:4147 , piece_id=BPNZTKMMEXM5B6UQSOLWZTZVVT5UOV2QHPSZL742DHQ3AQ3O6ESA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.283956314 (302h54m7.776388767s) [noise=true , address=45.91.116.6:4984 , piece_id=GQV5UTDPDAYNRXXQH7FL427GSQ5W5BLLOVQIN7NUIQIIIV7AJ4RQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284193911 (302h54m7.776626359s) [noise=true , address=145.79.227.13:4815 , piece_id=GBLGYGVE3WCY27XULJK6UYLUUZFPID33MA7X6N62TWHAMVRH3JBQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284732917 (302h54m7.777165362s) [noise=true , address=77.90.31.11:4556 , piece_id=QCYMCO46JRT2K6H46FYCDTZH6RTTL2Q46HOC2VW44IPTZ2W2KQRQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.284833471 (302h54m7.77726592s) [noise=true , address=145.79.227.33:4884 , piece_id=ZTHSKPCM7ZF5BKF26VRUXLJ2N3KGZYFK6HJTKOHYDUJFFYRR7HDA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.285523649 (302h54m7.777956091s) [noise=true , address=145.79.227.32:4854 , piece_id=2YLGSMO4KJ4OTLCQLTFWB6SFEEUPT3X5XKWE7KDQSX7W5LO7FO5Q, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.285585257 (302h54m7.778017699s) [noise=true , address=145.79.227.12:4770 , piece_id=NYBMVQLHBRCGAK2QQHSZ2BRZN3XAMMJKCRYKS4QWI7ZFH64XEGEA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.286297044 (302h54m7.778729499s) [noise=true , address=145.79.227.30:4771 , piece_id=SKXNQMX4RW3KCWKY5TMDPUI5QZJXYROJBMV5Y4P7MAUZUGCSUIEA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.286526017 (302h54m7.778958471s) [noise=true , address=145.79.227.5:4240 , piece_id=Q4PEISD7R7IO4BYBWLADEXE6JBUGQ5AY4J3IPX4P3Q6HZBUSYN7Q, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287104751 (302h54m7.779537193s) [noise=true , address=45.91.116.6:4008 , piece_id=PWJH5FFOUVPV64I4FLTGTBCMYLMIYBEZBZBUCI2WJQBKOPR4KABA, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287470770 (302h54m7.779903218s) [noise=true , address=145.79.227.17:4984 , piece_id=ARCUIGNVITSNKFJYAJQEP24HJVBBBDOZ3FRLUQASEZFN2IXGMPLQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287613969 (302h54m7.780046423s) [noise=true , address=145.79.227.14:4843 , piece_id=IXNCU3WCMDYAVT6IE4ETEMTKQROPHV4V77VO5TNNZP5ASMW3WJQQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.287960566 (302h54m7.780393019s) [noise=true , address=77.90.31.17:4795 , piece_id=2GUT5J77OOFK3OU6DG5IVXAXCBSV7SZRCXX3TPYZIRMCKGX3JUOQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289210694 (302h54m7.781643146s) [noise=true , address=145.79.227.11:4733 , piece_id=FPN5NMMTUYQHWRQOOYB5RPDKTWHEEWA3CX2OMH233YHTID6MWIFQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289586052 (302h54m7.782018502s) [noise=true , address=145.79.227.37:4589 , piece_id=K4HCLE6PM54ZXJVN5KO626R5UUP32OVEN6DDYIFUM6EYQCRVT7YQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.289971344 (302h54m7.782403791s) [noise=true , address=145.79.227.36:4524 , piece_id=RSUO6BNALYS7M6W4RJ55UOD35LNCPD26PQDZBCNY6HHVDFQTRVRQ, seg_pos={0 0}, upload=266387]: Uploading piece...
Sep 11 05:01:47.849451002 (302h54m8.341883437s) [upload=266387]: Next returned nil segment
Sep 11 05:01:47.850308233 (302h54m8.342740675s) [upload=266387]: Waiting for error group managing segments...
Sep 11 05:01:47.994127497 (302h54m8.486559926s) [noise=true , address=145.79.227.14:4843 , piece_id=IXNCU3WCMDYAVT6IE4ETEMTKQROPHV4V77VO5TNNZP5ASMW3WJQQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.995677280 (302h54m8.488109717s) [noise=true , address=145.79.227.11:4733 , piece_id=FPN5NMMTUYQHWRQOOYB5RPDKTWHEEWA3CX2OMH233YHTID6MWIFQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.996836577 (302h54m8.489269006s) [noise=true , address=145.79.227.30:4771 , piece_id=SKXNQMX4RW3KCWKY5TMDPUI5QZJXYROJBMV5Y4P7MAUZUGCSUIEA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.997314105 (302h54m8.489746535s) [noise=true , address=145.79.227.13:4815 , piece_id=GBLGYGVE3WCY27XULJK6UYLUUZFPID33MA7X6N62TWHAMVRH3JBQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.997716103 (302h54m8.490148547s) [noise=true , address=145.79.227.36:4524 , piece_id=RSUO6BNALYS7M6W4RJ55UOD35LNCPD26PQDZBCNY6HHVDFQTRVRQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.998369140 (302h54m8.490801569s) [noise=true , address=145.79.227.17:4984 , piece_id=ARCUIGNVITSNKFJYAJQEP24HJVBBBDOZ3FRLUQASEZFN2IXGMPLQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:47.998606161 (302h54m8.491038597s) [noise=true , address=77.90.31.11:4556 , piece_id=QCYMCO46JRT2K6H46FYCDTZH6RTTL2Q46HOC2VW44IPTZ2W2KQRQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.002273466 (302h54m8.494705902s) [noise=true , address=77.90.31.17:4795 , piece_id=2GUT5J77OOFK3OU6DG5IVXAXCBSV7SZRCXX3TPYZIRMCKGX3JUOQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.006810730 (302h54m8.499243159s) [noise=true , address=145.79.227.12:4770 , piece_id=NYBMVQLHBRCGAK2QQHSZ2BRZN3XAMMJKCRYKS4QWI7ZFH64XEGEA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.011564887 (302h54m8.503997331s) [noise=true , address=145.79.227.33:4884 , piece_id=ZTHSKPCM7ZF5BKF26VRUXLJ2N3KGZYFK6HJTKOHYDUJFFYRR7HDA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.023440869 (302h54m8.515873298s) [noise=true , address=45.91.116.11:4147 , piece_id=BPNZTKMMEXM5B6UQSOLWZTZVVT5UOV2QHPSZL742DHQ3AQ3O6ESA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:01:48.085368080 (302h54m8.577800508s) [noise=true , address=145.79.227.37:4589 , piece_id=K4HCLE6PM54ZXJVN5KO626R5UUP32OVEN6DDYIFUM6EYQCRVT7YQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: <nil>
Sep 11 05:02:46.754779889 (302h55m7.247212318s) [noise=true , address=145.79.227.5:4240 , piece_id=Q4PEISD7R7IO4BYBWLADEXE6JBUGQ5AY4J3IPX4P3Q6HZBUSYN7Q, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:13SaC5StbguPT3abAT9oTX3mZAyGhJcpAA6b7ueyiDFHa8CCEo, address:145.79.227.5:4240): protocol: rpc: dial tcp 145.79.227.5:4240: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.754831971 (302h55m7.247264401s) [noise=true , address=145.79.227.2:4121 , piece_id=KUVJMQVN5VOZCGW5W4DY5BMLI5IH23GSC2IFN6PH6XEA4OT3VUMA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:1Fq1JNvrZLTFTQLVF8jzyavJ6dU1xX96FTGcyxfaHiPdCLu4pB, address:145.79.227.2:4121): protocol: context deadline exceeded; context deadline exceeded
Sep 11 05:02:46.754895261 (302h55m7.247327705s) [noise=true , address=45.91.116.8:4051 , piece_id=PPCINK7K25XXRN3JQPMWVBQVPXXP7UPVSILUGIKWO3OI7EEI42FQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:1preT33ei4yVXXYVLHhaUbHbiyL14oHFEiof6cCWPig83W6CT3, address:45.91.116.8:4051): protocol: write tcp 139.59.144.22:38826->45.91.116.8:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.754988069 (302h55m7.247420503s) [noise=true , address=45.91.116.6:4008 , piece_id=PWJH5FFOUVPV64I4FLTGTBCMYLMIYBEZBZBUCI2WJQBKOPR4KABA, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:16KXs6ZXZyroRJibAYU5bsga1RFSpmf7yEtaNG7tamaX7QKvng, address:45.91.116.6:4008): protocol: context deadline exceeded; context deadline exceeded
Sep 11 05:02:46.755109851 (302h55m7.24754228s) [noise=true , address=45.91.116.6:4984 , piece_id=GQV5UTDPDAYNRXXQH7FL427GSQ5W5BLLOVQIN7NUIQIIIV7AJ4RQ, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt, address:45.91.116.6:4984): protocol: rpc: dial tcp 45.91.116.6:4984: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.755262600 (302h55m7.247695031s) [noise=true , address=145.79.227.32:4854 , piece_id=2YLGSMO4KJ4OTLCQLTFWB6SFEEUPT3X5XKWE7KDQSX7W5LO7FO5Q, seg_pos={0 0}, upload=266387]: Done uploading piece. err: ecclient: upload failed (node:12eitcwvcNX2GyMiv4GdfDD8wmgb6a1d3ggaGVxbpyKAjPjshmV, address:145.79.227.32:4854): protocol: rpc: dial tcp 145.79.227.32:4854: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Sep 11 05:02:46.755312278 (302h55m7.247744733s) [seg_pos={0 0}, upload=266387]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Sep 11 05:02:46.866353400 (302h55m7.358785843s) [upload=266387]: Done uploading segments...
Sep 11 05:02:46.866397083 (302h55m7.358829529s) [upload=266387]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 2.422044745s
core
uplink
us-select-1
us1-placement-20
|