Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.X.X........................................................] duration:
2.877134151s
FAIL (2025-07-09 20:57:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.877159044s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..X..............................................XX.........] duration:
25.253341588s
FAIL (2025-07-09 20:57:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 25.25344648s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
37.856395137s
FAIL (2025-07-09 20:57:10 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil>
Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload
Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload
Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments...
Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false
Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment...
Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment.
Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment
Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments...
Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000
Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments...
Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 37.8564174s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.41088048s
FAIL (2025-07-09 20:56:44 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments...
Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil>
Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload
Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload
Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments...
Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false
Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment...
Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment.
Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment
Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments...
Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces.
Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF
Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection
Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection
Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments...
Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil>
Duration: 2.410948948s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
21.692665785s
FAIL (2025-07-09 20:56:32 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil>
Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload
Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload
Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments...
Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false
Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment...
Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment.
Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment
Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments...
Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000
Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments...
Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 21.692699667s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X.X.......................................................] duration:
4.913489101s
FAIL (2025-07-09 20:56:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.913649879s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...X..............................................XX........] duration:
31.279062652s
FAIL (2025-07-09 20:56:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 31.279081342s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
20.502955663s
FAIL (2025-07-09 20:56:10 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil>
Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload
Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload
Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments...
Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false
Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment...
Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment.
Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment
Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments...
Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000
Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments...
Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 20.503001428s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [....X..............................................XX.......] duration:
38.484567925s
FAIL (2025-07-09 20:55:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 38.484612042s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.29675938s
FAIL (2025-07-09 20:55:50 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments...
Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil>
Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload
Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload
Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments...
Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false
Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment...
Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment.
Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment
Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments...
Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces.
Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF
Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection
Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection
Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments...
Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil>
Duration: 2.296789125s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
12.268023611s
FAIL (2025-07-09 20:55:41 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil>
Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload
Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload
Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments...
Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false
Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment...
Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment.
Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment
Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments...
Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000
Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments...
Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 12.268051504s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
43.289706794s
FAIL (2025-07-09 20:55:29 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil>
Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload
Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload
Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments...
Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false
Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment...
Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment.
Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment
Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments...
Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000
Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments...
Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 43.289728734s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...X.X......................................................] duration:
2.674218046s
FAIL (2025-07-09 20:55:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.67425265s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....X.X.....................................................] duration:
6.628921346s
FAIL (2025-07-09 20:55:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.628952657s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.551051823s
FAIL (2025-07-09 20:55:05 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments...
Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil>
Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload
Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload
Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments...
Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false
Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment...
Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment.
Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment
Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments...
Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces.
Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF
Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection
Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection
Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments...
Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil>
Duration: 2.551100406s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....X.X....................................................] duration:
2.637112024s
FAIL (2025-07-09 20:54:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.637143805s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.109486678s
FAIL (2025-07-09 20:54:46 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil>
Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload
Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload
Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments...
Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false
Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment...
Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment.
Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece...
Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment
Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments...
Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000
Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil>
Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments...
Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 1m0.109511678s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.....X..............................................XX......] duration:
35.674497206s
FAIL (2025-07-09 20:54:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 35.674510715s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.353473822s
FAIL (2025-07-09 20:54:19 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments...
Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil>
Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload
Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload
Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments...
Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false
Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment...
Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment.
Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment
Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments...
Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces.
Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF
Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection
Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection
Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments...
Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil>
Duration: 2.353505054s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
15.582281221s
FAIL (2025-07-09 20:54:01 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments...
Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil>
Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload
Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload
Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments...
Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false
Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment...
Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment.
Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment
Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments...
Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces.
Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF
Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection
Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection
Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments...
Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil>
Duration: 15.582340787s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [......X..............................................XX.....] duration:
55.135978099s
FAIL (2025-07-09 20:54:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 55.136003435s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......X.X...................................................] duration:
2.686613912s
FAIL (2025-07-09 20:53:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.686654559s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.312884941s
FAIL (2025-07-09 20:53:08 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments...
Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil>
Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload
Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload
Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments...
Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false
Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment...
Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment.
Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment
Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments...
Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces.
Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF
Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection
Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection
Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments...
Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil>
Duration: 2.312916527s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......X.X..................................................] duration:
2.626876202s
FAIL (2025-07-09 20:53:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.6269054s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.......X..............................................XX....] duration:
36.025669406s
FAIL (2025-07-09 20:52:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 36.025692641s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........X.X.................................................] duration:
2.516975557s
FAIL (2025-07-09 20:52:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.517007481s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
3.950915092s
FAIL (2025-07-09 20:52:08 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments...
Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil>
Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload
Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload
Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments...
Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false
Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment...
Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment.
Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment
Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments...
Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces.
Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF
Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection
Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection
Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments...
Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil>
Duration: 3.950991609s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........X.X................................................] duration:
2.915823425s
FAIL (2025-07-09 20:52:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.915852771s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [X.......X..............................................XX...] duration:
44.470009191s
FAIL (2025-07-09 20:52:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 44.470026487s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
2.303724305s
FAIL (2025-07-09 20:51:16 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments...
Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil>
Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload
Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload
Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments...
Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false
Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment...
Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment.
Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment
Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments...
Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces.
Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF
Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection
Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection
Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments...
Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil>
Duration: 2.303752432s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........X.X...............................................] duration:
2.633459837s
FAIL (2025-07-09 20:51:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.63347593s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.X.......X..............................................XX..] duration:
18.010701075s
FAIL (2025-07-09 20:50:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 18.010796809s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........X.X..............................................] duration:
2.822433901s
FAIL (2025-07-09 20:50:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.822467205s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [............X.X.............................................] duration:
2.97535597s
FAIL (2025-07-09 20:50:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.975383212s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.453998344s
FAIL (2025-07-09 20:50:07 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments...
Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil>
Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload
Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload
Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments...
Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false
Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment...
Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment.
Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment
Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments...
Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces.
Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF
Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection
Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection
Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments...
Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil>
Duration: 2.454027954s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [X.X.......X..............................................XX.] duration:
23.425753414s
FAIL (2025-07-09 20:49:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 23.426177518s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............X.X............................................] duration:
2.779574791s
FAIL (2025-07-09 20:49:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.779606247s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.28232202s
FAIL (2025-07-09 20:49:37 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments...
Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil>
Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload
Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload
Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments...
Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false
Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment...
Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment.
Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment
Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments...
Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces.
Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF
Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection
Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection
Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments...
Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil>
Duration: 2.282350747s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.X.X.......X..............................................XX] duration:
1m0.001909927s
FAIL (2025-07-09 20:49:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001928993s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............X.X...........................................] duration:
3.032298754s
FAIL (2025-07-09 20:49:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.032327989s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............X.X..........................................] duration:
2.812178463s
FAIL (2025-07-09 20:48:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.812487439s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.356304678s
FAIL (2025-07-09 20:48:37 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments...
Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil>
Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload
Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload
Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments...
Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false
Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment...
Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment.
Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment
Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments...
Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces.
Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF
Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection
Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection
Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments...
Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil>
Duration: 2.356330429s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.003300717s
FAIL (2025-07-09 20:48:35 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments...
Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil>
Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload
Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload
Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments...
Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false
Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment...
Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment.
Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece...
Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment
Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments...
Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil>
Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces.
Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF
Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection
Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection
Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments...
Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil>
Duration: 30.003336597s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [................X.X.........................................] duration:
2.661737143s
FAIL (2025-07-09 20:48:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.661769752s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..X.X.......X..............................................X] duration:
1m0.000961661s
FAIL (2025-07-09 20:48:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.00097529s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...X.X.......X..............................................] duration:
32.641236845s
FAIL (2025-07-09 20:47:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 32.641255242s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................X.X........................................] duration:
2.564645922s
FAIL (2025-07-09 20:47:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.564671459s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [....X.X.......X.............................................] duration:
23.059621856s
FAIL (2025-07-09 20:46:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 23.059649756s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................X.X.......................................] duration:
2.776703687s
FAIL (2025-07-09 20:46:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.776725733s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.....X.X.......X............................................] duration:
28.772794871s
FAIL (2025-07-09 20:46:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 28.772810647s
core
uplink
us-select-1
us1-placement-20
|