Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 56.765040119s FAIL (2025-07-09 20:01:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 56.765067737s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 20.29343305s FAIL (2025-07-09 20:00:21 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 20.293475924s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 25.880986491s FAIL (2025-07-09 20:00:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 25.881012313s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................X............X................] duration: 2.667594041s FAIL (2025-07-09 19:59:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.667619966s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 28.592028668s FAIL (2025-07-09 19:59:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 28.592049988s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X............X...............] duration: 2.730273059s FAIL (2025-07-09 19:59:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.730299957s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 7.721649461s FAIL (2025-07-09 19:58:56 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 7.721677542s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 42.184178829s FAIL (2025-07-09 19:58:44 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 42.184270861s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X............X..............] duration: 2.711318368s FAIL (2025-07-09 19:58:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.711363932s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 23.133215039s FAIL (2025-07-09 19:58:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 23.133243374s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X............X.............] duration: 2.749900518s FAIL (2025-07-09 19:57:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.749929675s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 14.709528174s FAIL (2025-07-09 19:57:17 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 14.709560253s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X............X............] duration: 2.869432017s FAIL (2025-07-09 19:57:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.869460829s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [X...................................X.......................] duration: 8.246351355s FAIL (2025-07-09 19:57:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.246386986s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 26.449461862s FAIL (2025-07-09 19:56:47 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 26.449490994s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.X...................................X......................] duration: 21.82042657s FAIL (2025-07-09 19:56:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 21.820452071s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X............X...........] duration: 2.780721325s FAIL (2025-07-09 19:56:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.780751414s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..X...................................X.....................] duration: 32.917063246s FAIL (2025-07-09 19:55:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 32.917085296s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 9.547322047s FAIL (2025-07-09 19:55:26 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 9.547383484s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 21.177714959s FAIL (2025-07-09 19:55:16 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 21.177740753s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X............X..........] duration: 2.864355311s FAIL (2025-07-09 19:55:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.864384312s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 33.506544269s FAIL (2025-07-09 19:54:55 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 33.506575585s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...X...................................X....................] duration: 27.377903018s FAIL (2025-07-09 19:54:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 27.37792261s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X............X.........] duration: 3.048260101s FAIL (2025-07-09 19:54:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.04829316s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....X...................................X...................] duration: 55.440689617s FAIL (2025-07-09 19:54:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 55.440830672s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X............X........] duration: 2.727560815s FAIL (2025-07-09 19:54:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.727587428s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 32.103671424s FAIL (2025-07-09 19:53:45 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 32.103697299s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X............X.......] duration: 2.685195546s FAIL (2025-07-09 19:53:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.685219669s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....X...................................X..................] duration: 46.094874426s FAIL (2025-07-09 19:53:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 46.094910784s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X............X......] duration: 2.861714717s FAIL (2025-07-09 19:53:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.86174492s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 25.40229146s FAIL (2025-07-09 19:52:38 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 25.40232452s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X............X.....] duration: 2.811789026s FAIL (2025-07-09 19:52:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.811815554s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X............X....] duration: 2.781256389s FAIL (2025-07-09 19:51:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.7812835s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......X...................................X.................] duration: 25.860235952s FAIL (2025-07-09 19:51:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 25.860353435s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 21.370945071s FAIL (2025-07-09 19:51:30 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 21.370974667s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X............X...] duration: 2.653103807s FAIL (2025-07-09 19:51:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.653140938s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X............X..] duration: 2.802428511s FAIL (2025-07-09 19:51:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.802469833s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......X...................................X................] duration: 25.997456786s FAIL (2025-07-09 19:50:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 25.997472805s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 12.105135825s FAIL (2025-07-09 19:50:18 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 12.105161928s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........X...................................X...............] duration: 41.761893161s FAIL (2025-07-09 19:50:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 41.761923086s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X............X.] duration: 2.874000412s FAIL (2025-07-09 19:50:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.874073283s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X............X] duration: 30.001121039s FAIL (2025-07-09 19:49:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001161944s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 14.10134605s FAIL (2025-07-09 19:49:24 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 14.10137533s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........X...................................X..............] duration: 21.942322146s FAIL (2025-07-09 19:49:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 21.942417655s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 28.632526758s FAIL (2025-07-09 19:49:05 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 28.632550932s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.900834786s FAIL (2025-07-09 19:48:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.900860811s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 29.793690454s FAIL (2025-07-09 19:48:36 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 07.775154767 (394h29m7.658568397s) [upload=315464]: Got next segment. Inline: false Jul 9 19:40:07.775222675 (394h29m7.658636302s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 19:40:07.775228517 (394h29m7.658642145s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 19:40:08.077785366 (394h29m7.961199002s) [seg_pos={0 0}, upload=315464]: Begin upload segment... Jul 9 19:40:08.077860847 (394h29m7.961274483s) [seg_pos={0 0}, upload=315464]: Done begin upload segment. Jul 9 19:40:08.077958256 (394h29m7.961371896s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.077986061 (394h29m7.961399693s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.078965751 (394h29m7.962379387s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079037038 (394h29m7.962450674s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.079207315 (394h29m7.962620951s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080140241 (394h29m7.963553877s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.080378066 (394h29m7.963791701s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081072970 (394h29m7.964486607s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081177519 (394h29m7.964591157s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.081889239 (394h29m7.965302875s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082087416 (394h29m7.965501052s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.082453979 (394h29m7.965867615s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083004219 (394h29m7.966417856s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.083371902 (394h29m7.966785539s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084191128 (394h29m7.967604765s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084571607 (394h29m7.967985244s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.084912151 (394h29m7.968325788s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.085249687 (394h29m7.968663324s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Uploading piece... Jul 9 19:40:08.616577029 (394h29m8.499990646s) [noise=true , address=79.127.201.214:4306 , piece_id=U5DCBRADG2OHKAOKUL6FOUHFVYI2XRXUWXDFNBET4URYOLC5SHNQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: ecclient: upload failed (node:122DJmb292J722vUUNey4itWFfZNT35ruZpjz4awcfcU3QEmQLc, address:79.127.201.214:4306): protocol: storage node overloaded, request limit: 2000 Jul 9 19:40:08.660838400 (394h29m8.544252021s) [upload=315464]: Next returned nil segment Jul 9 19:40:08.660853085 (394h29m8.544266704s) [upload=315464]: Waiting for error group managing segments... Jul 9 19:40:08.800543989 (394h29m8.68395762s) [noise=true , address=77.90.31.15:4718 , piece_id=Y3FJTNURRRK4AUVWGEXIAVYPBLCMT5WS6DN3KNJONBXAYVRU2WKQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.802597777 (394h29m8.686011397s) [noise=true , address=77.90.31.10:4531 , piece_id=KOVJIOT5GELWIFQWAZMX6HMFIADYSL7TCHHBCDQZHDJCP6HCIZPA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.805684069 (394h29m8.689097688s) [noise=true , address=77.90.31.10:4529 , piece_id=C3EDANWA3D3PLXDJQN5KYUHEVYVWAUE2OMBX33UII3UFTA3DOPRQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.806282081 (394h29m8.689695701s) [noise=true , address=77.90.31.13:4637 , piece_id=34WA2VCHAQUVZXH7RISTGWMTBUYDQ5YBHGQSVXJQXEZF7EXXLUIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814098678 (394h29m8.697512311s) [noise=true , address=77.90.31.10:4524 , piece_id=6YJIRRJDCIBUTRL4YXB27SYRPI6XGKWYKUAWNCGCON7BDHSIBTXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:08.814485406 (394h29m8.69789902s) [noise=true , address=77.90.31.18:4809 , piece_id=BIRC5T5KEYC2B6G73OZDDKCSUCXNZQ6UMKNHTJOOXQ5BFJW3OTIQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:10.066359268 (394h29m9.949772895s) [noise=true , address=156.146.43.225:4441 , piece_id=J6LFPF7F4RLBZRCESSUL4C7I7KWSWXGEC6YCHABJ63YP2LOJNBIA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:12.201742629 (394h29m12.085156256s) [noise=true , address=79.127.220.98:4413 , piece_id=NA7XBGOZCFXP3YF2LKCKA77FBT55LUZ6W6X4UQYYZLL45BEIS3XA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:13.472536969 (394h29m13.355950596s) [noise=true , address=45.91.116.6:4997 , piece_id=DKHOFAVZ3WXCOCUJSIQ7FSTN4Y56XZUN7MZABYPGKSBREERKMBFA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:16.917628152 (394h29m16.801041778s) [noise=true , address=45.91.116.9:4266 , piece_id=OZY7YEVACKF2HZIIOMF5MERWLJ63RQFSUVXPZAMZDXIDP5EKEZHQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:17.483834714 (394h29m17.367248345s) [noise=true , address=104.36.82.13:4192 , piece_id=5NTMTBKHCGITRBPDZKOS2T45ZZPVL77D4VLR4BQNQZA57AH4FIPQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:25.809751434 (394h29m25.693165062s) [noise=true , address=104.36.82.17:4337 , piece_id=SRZAQHNA2ULRQBJNOUC5N4VNHE2QXKCLLK4FLFH72LLXRZD3RH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:40:59.183773427 (394h29m59.067187039s) [noise=true , address=104.36.82.44:4883 , piece_id=C5NO3GCKQ4XQKPY54N2BJRNIEPC6527FSQKR6MB6XF5WBBCBVDYQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028555873 (394h30m3.911969493s) [noise=true , address=104.36.82.19:4426 , piece_id=JU7DFVYHFVGUHHD4ATGUSEMCGN5CTEJNQCZ7L5NRCMTR7JCUEBXQ, seg_pos={0 0}, upload=315464]: Done uploading piece. err: <nil> Jul 9 19:41:04.028586121 (394h30m3.911999747s) [seg_pos={0 0}, upload=315464]: Segment reached optimal threshold of 14 pieces. Jul 9 19:41:04.028700106 (394h30m3.912113722s) [noise=true , address=104.36.82.5:4864 , piece_id=MQFCBTUC4VPHQN7UQXCOKY2WRT2WKTNGNPNHTFKOJIWIH5JXQJ7A, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028743454 (394h30m3.91215708s) [noise=true , address=104.36.82.36:4642 , piece_id=AXEMFG7WBJ57Z5WPC5RTFNDOITUDNIKX3UVIVUOTB3VBX63EJ4MA, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127tVQUAwStYLonZ1jjq2tjtuqbLpKh593Uj1yyhGPy5XQ5AF5L): protocol: expected piece hash; context canceled; EOF Jul 9 19:41:04.028831009 (394h30m3.912244626s) [noise=true , address=104.36.82.21:4493 , piece_id=BFUQHSHN6ZVT33BYZAYTGD6SYYZLUG77CEHDDAIYDJBB7NOWAH3Q, seg_pos={0 0}, upload=315464]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUjzJu7JJ4q77xnkBEZtPV2bwg5UJtMBfkgnjuwdAHc7Ff5UD): protocol: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58658->104.36.82.21:4493: use of closed network connection Jul 9 19:41:04.028844681 (394h30m3.912258299s) [seg_pos={0 0}, upload=315464]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 9 19:41:04.028918760 (394h30m3.912332377s) [upload=315464]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 19:41:04.028923227 (394h30m3.912336845s) [upload=315464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 19:41:04.186917131 (394h30m4.070330747s) [upload=315464]: Done uploading segments... Jul 9 19:41:04.186938057 (394h30m4.070351676s) [upload=315464]: Upload finished. err: <nil> Duration: 29.793718834s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........X...................................X.............] duration: 28.845976365s FAIL (2025-07-09 19:48:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 28.845993831s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........X...................................X............] duration: 40.806708116s FAIL (2025-07-09 19:47:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 40.806732052s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.579029292s FAIL (2025-07-09 19:47:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.579052563s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)