Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.760283862s
FAIL (2025-10-06 22:53:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.760404685s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
25.178060321s
FAIL (2025-10-06 22:52:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
3.266647152 (321h21m51.070111746s) [upload=282801]: Starting upload
Oct 6 22:52:03.266773242 (321h21m51.070237837s) [upload=282801]: Done starting upload
Oct 6 22:52:03.266937035 (321h21m51.070401633s) [upload=282801]: Uploading segments...
Oct 6 22:52:03.267176514 (321h21m51.070641227s) [upload=282801]: Got next segment. Inline: false
Oct 6 22:52:03.268089244 (321h21m51.071553852s) [upload=282801]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:52:03.268111522 (321h21m51.071576123s) [upload=282801]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:52:03.430037228 (321h21m51.233501827s) [seg_pos={0 0}, upload=282801]: Begin upload segment...
Oct 6 22:52:03.430113862 (321h21m51.233578456s) [seg_pos={0 0}, upload=282801]: Done begin upload segment.
Oct 6 22:52:03.430191012 (321h21m51.233655604s) [noise=true , address=145.79.227.5:4223 , piece_id=IZPHPGP4JPK6CYPXZB3O7AEZDR4CHGOSWHK25N45WVRXAWLH4IGQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.430232477 (321h21m51.233697077s) [noise=true , address=145.79.227.3:4151 , piece_id=3FOVZXVEB3253MLNM4XXQXB2MVTOKK3XJZR6H55SDJ4IEUBLLHDQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.430824279 (321h21m51.234288885s) [noise=true , address=77.90.31.17:4770 , piece_id=GFMQGGGYDEUAJNZSMFJHTP5PXEC537E5AYUCRSMNQ674WVZY5TGA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.434338118 (321h21m51.23780271s) [noise=true , address=77.90.31.19:4841 , piece_id=ALRINMNMX7CYVWPIVSVRDPLHIHGZ2VUIGZLRGI5HTRPA5IEDMEAQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.435142913 (321h21m51.238607514s) [noise=true , address=45.91.116.9:4262 , piece_id=W4OXKGKAE35KCKDSNQL3S75QZPIONV5GLOVACWIC3M37ME5RBAAQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.435954054 (321h21m51.239418669s) [noise=true , address=145.79.227.14:4839 , piece_id=SVISSTD52AGG7E7JKNW5HV4ZAIHMQ3CD5PKAGY2O4AYK4PA2PWDQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.436658415 (321h21m51.240123013s) [noise=true , address=145.79.227.17:4969 , piece_id=PAOB6PTNPO7SJP327WRVCJ2STRQQMIJEVLSCGWCAFRXRLRZAV44A, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.437196793 (321h21m51.240661376s) [noise=true , address=77.90.31.11:4553 , piece_id=MGSQPY7RJXHKO7UVWEHGF7ND26PWKEXSSEGXDTH2C3ESIKD436MA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.437784701 (321h21m51.241249295s) [noise=true , address=77.90.31.10:4518 , piece_id=3MSB7KIKPCAFCHUI3UOCYFAEYG7W4I2A4I7W7MAOX6YXHVCKURUA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.438473814 (321h21m51.24193841s) [noise=true , address=45.91.116.12:4227 , piece_id=FLFWRGBERDRKWNZ4V4FXYCZOIPQULHGE3NYOSR2KL3TDPPC6Z4NA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.446137710 (321h21m51.249602322s) [noise=true , address=145.79.227.5:4222 , piece_id=QSJ4JXWZ3BR5H7BXWWJTDZP2BL667L67VEPD3HJMCXAULRYLL6TA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.448705223 (321h21m51.252169836s) [noise=true , address=145.79.227.12:4767 , piece_id=6IPOWLQUCRC2EZOV24BTJV3UQ2W47MW7YFCIK4W3VAYVVFI44MFQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.453854771 (321h21m51.257319379s) [noise=true , address=145.79.227.16:4910 , piece_id=HOCU23QR3XDR7Z5CXAY7J6E3D7DI4XMG5TDJCPVLPXIYZFIFFFQA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.462270326 (321h21m51.265734926s) [noise=true , address=145.79.227.18:4006 , piece_id=46DN34BCNPULTMOKSC2ECHPAZIB53LXIPLM2DWOKL56WJEWCEL4A, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.464373362 (321h21m51.267837968s) [noise=true , address=45.91.116.11:4119 , piece_id=2AYCBJSFSSYYRGFEMR5UWT3OLIEEWXEZVTIUU4N6OUWRSVPIM7YQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.465236516 (321h21m51.26870112s) [noise=true , address=145.79.227.3:4150 , piece_id=4LD34HU6CUUC67PRQ2ZA7SC2DCAVMKKAXZECKTEB6U4D7VGJOYHA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:04.132660648 (321h21m51.936125282s) [upload=282801]: Next returned nil segment
Oct 6 22:52:04.132711626 (321h21m51.936176233s) [upload=282801]: Waiting for error group managing segments...
Oct 6 22:52:04.268577053 (321h21m52.072041634s) [noise=true , address=145.79.227.14:4839 , piece_id=SVISSTD52AGG7E7JKNW5HV4ZAIHMQ3CD5PKAGY2O4AYK4PA2PWDQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.269076811 (321h21m52.072541395s) [noise=true , address=145.79.227.3:4151 , piece_id=3FOVZXVEB3253MLNM4XXQXB2MVTOKK3XJZR6H55SDJ4IEUBLLHDQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.269136691 (321h21m52.072601272s) [noise=true , address=145.79.227.12:4767 , piece_id=6IPOWLQUCRC2EZOV24BTJV3UQ2W47MW7YFCIK4W3VAYVVFI44MFQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.276452785 (321h21m52.079917379s) [noise=true , address=145.79.227.5:4222 , piece_id=QSJ4JXWZ3BR5H7BXWWJTDZP2BL667L67VEPD3HJMCXAULRYLL6TA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.279150664 (321h21m52.082615249s) [noise=true , address=145.79.227.17:4969 , piece_id=PAOB6PTNPO7SJP327WRVCJ2STRQQMIJEVLSCGWCAFRXRLRZAV44A, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.282533290 (321h21m52.085997871s) [noise=true , address=145.79.227.5:4223 , piece_id=IZPHPGP4JPK6CYPXZB3O7AEZDR4CHGOSWHK25N45WVRXAWLH4IGQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.286605407 (321h21m52.090069991s) [noise=true , address=145.79.227.16:4910 , piece_id=HOCU23QR3XDR7Z5CXAY7J6E3D7DI4XMG5TDJCPVLPXIYZFIFFFQA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.291130118 (321h21m52.094594711s) [noise=true , address=145.79.227.18:4006 , piece_id=46DN34BCNPULTMOKSC2ECHPAZIB53LXIPLM2DWOKL56WJEWCEL4A, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.296637658 (321h21m52.100102253s) [noise=true , address=145.79.227.3:4150 , piece_id=4LD34HU6CUUC67PRQ2ZA7SC2DCAVMKKAXZECKTEB6U4D7VGJOYHA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:32.905984135 (321h22m20.709448735s) [noise=true , address=45.91.116.9:4262 , piece_id=W4OXKGKAE35KCKDSNQL3S75QZPIONV5GLOVACWIC3M37ME5RBAAQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 139.59.144.22:48884->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906132393 (321h22m20.709596977s) [noise=true , address=77.90.31.17:4770 , piece_id=GFMQGGGYDEUAJNZSMFJHTP5PXEC537E5AYUCRSMNQ674WVZY5TGA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906293048 (321h22m20.70975764s) [noise=true , address=77.90.31.10:4518 , piece_id=3MSB7KIKPCAFCHUI3UOCYFAEYG7W4I2A4I7W7MAOX6YXHVCKURUA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: read tcp 139.59.144.22:59290->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906302895 (321h22m20.709767512s) [noise=true , address=77.90.31.11:4553 , piece_id=MGSQPY7RJXHKO7UVWEHGF7ND26PWKEXSSEGXDTH2C3ESIKD436MA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:47872->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906403245 (321h22m20.709867837s) [noise=true , address=45.91.116.11:4119 , piece_id=2AYCBJSFSSYYRGFEMR5UWT3OLIEEWXEZVTIUU4N6OUWRSVPIM7YQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:49046->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906429852 (321h22m20.709894447s) [noise=true , address=77.90.31.19:4841 , piece_id=ALRINMNMX7CYVWPIVSVRDPLHIHGZ2VUIGZLRGI5HTRPA5IEDMEAQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:37584->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906437501 (321h22m20.709902093s) [noise=true , address=45.91.116.12:4227 , piece_id=FLFWRGBERDRKWNZ4V4FXYCZOIPQULHGE3NYOSR2KL3TDPPC6Z4NA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:37362->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906459259 (321h22m20.709923851s) [seg_pos={0 0}, upload=282801]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 6 22:52:33.017661298 (321h22m20.821125895s) [upload=282801]: Done uploading segments...
Oct 6 22:52:33.017697522 (321h22m20.821162119s) [upload=282801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 25.178085522s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.516365444s
FAIL (2025-10-06 22:52:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 6)
======LOGS======
4962 , piece_id=QPJBUB5IFOCI2WC2QHHNHVGE47U5BNTE2Y7CUKM7EDLOGAAAG4RA, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.837450923 (321h21m43.640915517s) [noise=true , address=64.49.13.155:4914 , piece_id=YB4BAA33YQTC4E6ZOGASTE7MDR46ZUJVNXK2VEMTPCZKIILSNNQA, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.837746517 (321h21m43.641211108s) [noise=true , address=45.91.116.11:4147 , piece_id=6U24K2NHSU2KJ4AH3YSM7ZLKQNKH55ASKFHN42W3PCZYWJS6VREA, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.838078923 (321h21m43.641543517s) [noise=true , address=77.90.31.12:4617 , piece_id=D2P2E5AH3QQRZD2MKT2UOUG7Y4QOARXSFU7MPBVQF2VFCMMYERCQ, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.839295036 (321h21m43.642759636s) [noise=true , address=77.90.31.17:4777 , piece_id=WCHVJ5MDVIGSQWE4VYSG2BRCIYFKGWBAZCQGDDVEEIO4ZIK4MGCQ, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.839737614 (321h21m43.643202212s) [noise=true , address=45.91.116.7:4028 , piece_id=4DJWSDSQ45GSWCZH6H6OVGFWLAXRH27QI4CQAUKJG7Y45KDOV7SA, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.841802119 (321h21m43.645266713s) [noise=true , address=45.91.116.13:4162 , piece_id=RLDKWRPWM3WX6O42BTHBL75LSBSPRBEVDYEHXDBUQ4IN3X5P3OTQ, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.843215985 (321h21m43.646680582s) [noise=true , address=145.79.227.44:4225 , piece_id=VMXBMMZYIF77KFOLGSUAQWXPLACW2MK32DA7VWWXIYZ6J5RG4GCQ, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.843918826 (321h21m43.647383423s) [noise=true , address=145.79.227.43:4171 , piece_id=BTVHWDDV4636C3KSKHJRQI3L7RSO5K46S7EBSOWIZJWOIGFKPY6Q, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.847034063 (321h21m43.650498653s) [noise=true , address=45.91.116.9:4284 , piece_id=TTL5JXNQ7KOIFBQU7HQOGWCDNSWG5ZFIOXEZI22WLZ326H7YO25A, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.849936736 (321h21m43.653401326s) [noise=true , address=45.91.116.5:4316 , piece_id=4V7EM5VI4OHF7CXLPK6EXHP2QUVYT42PK3BFT6243VZ42OBEYTKA, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.852057350 (321h21m43.655521951s) [noise=true , address=77.90.31.16:4763 , piece_id=G4RVFEBK4O4QTYBB5AMTL2V6UIF5SCWSCIASBTRDWXUJPINBAK6A, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:56.437181449 (321h21m44.240646041s) [upload=282798]: Next returned nil segment
Oct 6 22:51:56.437194503 (321h21m44.240659095s) [upload=282798]: Waiting for error group managing segments...
Oct 6 22:51:56.621774513 (321h21m44.425239091s) [noise=true , address=145.79.227.44:4225 , piece_id=VMXBMMZYIF77KFOLGSUAQWXPLACW2MK32DA7VWWXIYZ6J5RG4GCQ, seg_pos={0 0}, upload=282798]: Done uploading piece. err: <nil>
Oct 6 22:51:56.621931773 (321h21m44.425396351s) [noise=true , address=145.79.227.22:4519 , piece_id=YZRNLOAN5YBX734URL2EB2YCLXYVXATCHZKLGWCXOXX2QU63XRUQ, seg_pos={0 0}, upload=282798]: Done uploading piece. err: <nil>
Oct 6 22:51:56.632506254 (321h21m44.435970833s) [noise=true , address=145.79.227.31:4802 , piece_id=YPNDUTRTU7IYGARW2SXIUBEG5SOGWEWRZ4HC4YHD7BQPEC7JZ4UQ, seg_pos={0 0}, upload=282798]: Done uploading piece. err: <nil>
Oct 6 22:51:56.638511638 (321h21m44.441976227s) [noise=true , address=145.79.227.43:4171 , piece_id=BTVHWDDV4636C3KSKHJRQI3L7RSO5K46S7EBSOWIZJWOIGFKPY6Q, seg_pos={0 0}, upload=282798]: Done uploading piece. err: <nil>
Oct 6 22:51:56.645212964 (321h21m44.448677548s) [noise=true , address=145.79.227.30:4792 , piece_id=D3344KOSAB34R26FAEZYQMTHCCNOFJKQOTV46PGTDJEAKCZDGFIA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: <nil>
Oct 6 22:51:57.099414812 (321h21m44.902879399s) [noise=true , address=64.49.13.155:4914 , piece_id=YB4BAA33YQTC4E6ZOGASTE7MDR46ZUJVNXK2VEMTPCZKIILSNNQA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: <nil>
Oct 6 22:52:55.433094081 (321h22m43.236558673s) [noise=true , address=45.91.116.13:4162 , piece_id=RLDKWRPWM3WX6O42BTHBL75LSBSPRBEVDYEHXDBUQ4IN3X5P3OTQ, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:1geUSD7qjW2yuMayTJ1Zy1KTYVC19adZ9nUWMRtmg2TktJSsm4, address:45.91.116.13:4162): protocol: rpc: dial tcp 45.91.116.13:4162: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433274540 (321h22m43.236739125s) [noise=true , address=77.90.31.16:4763 , piece_id=G4RVFEBK4O4QTYBB5AMTL2V6UIF5SCWSCIASBTRDWXUJPINBAK6A, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:12Bn4bKB4U5JidFqnXuPTncmNPGi72tgbL5YmYBRbbJ6ckqo35d, address:77.90.31.16:4763): protocol: rpc: dial tcp 77.90.31.16:4763: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433441208 (321h22m43.236905798s) [noise=true , address=45.91.116.5:4316 , piece_id=4V7EM5VI4OHF7CXLPK6EXHP2QUVYT42PK3BFT6243VZ42OBEYTKA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:1viKEvjPFGZ9E6E9DFPNXZWYE6SYoh4gzxNYqGGK6Mt2q9YciQ, address:45.91.116.5:4316): protocol: read tcp 139.59.144.22:44382->45.91.116.5:4316: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433509768 (321h22m43.23697436s) [noise=true , address=77.90.31.18:4812 , piece_id=BZVDTBRIDDP4PYS553QMYDGGC442A5B4PGK7LBMJQ4RGMFIWQS5Q, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:128qNJJSSwMBPvN8wmTcUeMWAJxVmpjWjwA6WTKxBNCmT2FdFJn, address:77.90.31.18:4812): protocol: read tcp 139.59.144.22:49462->77.90.31.18:4812: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433558250 (321h22m43.237022839s) [noise=true , address=45.91.116.6:4986 , piece_id=M2HAEBACWUTJYNE5PVV7EZVH5KVERGUFCAEU5FODBHLSHOY5E2EA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:12t1cuEr8MRLBbJ4GNSdYBxdhNQ6xDbTr8PHoEuVhMqh1ksgWon, address:45.91.116.6:4986): protocol: read tcp 139.59.144.22:45000->45.91.116.6:4986: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433670048 (321h22m43.237134631s) [noise=true , address=77.90.31.12:4617 , piece_id=D2P2E5AH3QQRZD2MKT2UOUG7Y4QOARXSFU7MPBVQF2VFCMMYERCQ, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:47744->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433755331 (321h22m43.237219914s) [noise=true , address=45.91.116.8:4052 , piece_id=JJ472TZ53QOWOCOC45SNCEWQE76NJO4LM4NPB6TFK2FQWVKACB4A, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:1HHEiq4G4TCaHFJAN3DhL8SwXAFqcwRgYteSHmr1gNe2ThcmnS, address:45.91.116.8:4052): protocol: read tcp 139.59.144.22:33770->45.91.116.8:4052: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433781714 (321h22m43.237246302s) [noise=true , address=45.91.116.4:4962 , piece_id=QPJBUB5IFOCI2WC2QHHNHVGE47U5BNTE2Y7CUKM7EDLOGAAAG4RA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:115Q9FLfn9BcC5kHcdAWa7yn9myRVusdj12Qcy7nvLr1kUXdvF, address:45.91.116.4:4962): protocol: read tcp 139.59.144.22:40746->45.91.116.4:4962: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433897621 (321h22m43.237362213s) [noise=true , address=45.91.116.11:4147 , piece_id=6U24K2NHSU2KJ4AH3YSM7ZLKQNKH55ASKFHN42W3PCZYWJS6VREA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:12YnWetw1NereHQi7cwHCnmu1LDjRFNpB2MQHSWQrjQdN1CZbMw, address:45.91.116.11:4147): protocol: read tcp 139.59.144.22:49742->45.91.116.11:4147: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.434092748 (321h22m43.237557339s) [noise=true , address=77.90.31.17:4777 , piece_id=WCHVJ5MDVIGSQWE4VYSG2BRCIYFKGWBAZCQGDDVEEIO4ZIK4MGCQ, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:1DjhLCRQRkNECPQg8HG8kZcBn8jJmBZpNhdywxEK7ZUn1v19wM, address:77.90.31.17:4777): protocol: read tcp 139.59.144.22:41604->77.90.31.17:4777: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.434160736 (321h22m43.237625314s) [noise=true , address=45.91.116.9:4284 , piece_id=TTL5JXNQ7KOIFBQU7HQOGWCDNSWG5ZFIOXEZI22WLZ326H7YO25A, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:127CKUmRzR3215HNASZzQmbiy6shYsv98XjmQ2uGu4NjA37KKXx, address:45.91.116.9:4284): protocol: read tcp 139.59.144.22:55276->45.91.116.9:4284: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.434178177 (321h22m43.237642775s) [noise=true , address=45.91.116.7:4028 , piece_id=4DJWSDSQ45GSWCZH6H6OVGFWLAXRH27QI4CQAUKJG7Y45KDOV7SA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:1g3sbXJir88dVbuJCShqFZ5KH2RJhq1kSqfVx6wN7ZkQtucnrB, address:45.91.116.7:4028): protocol: write tcp 139.59.144.22:41728->45.91.116.7:4028: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.434211421 (321h22m43.237676012s) [seg_pos={0 0}, upload=282798]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil>
Oct 6 22:52:55.542478957 (321h22m43.345943538s) [upload=282798]: Done uploading segments...
Oct 6 22:52:55.542508398 (321h22m43.34597299s) [upload=282798]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6)
Duration: 2.516384973s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.110863895s
FAIL (2025-10-06 22:52:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 6)
======LOGS======
4962 , piece_id=QPJBUB5IFOCI2WC2QHHNHVGE47U5BNTE2Y7CUKM7EDLOGAAAG4RA, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.837450923 (321h21m43.640915517s) [noise=true , address=64.49.13.155:4914 , piece_id=YB4BAA33YQTC4E6ZOGASTE7MDR46ZUJVNXK2VEMTPCZKIILSNNQA, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.837746517 (321h21m43.641211108s) [noise=true , address=45.91.116.11:4147 , piece_id=6U24K2NHSU2KJ4AH3YSM7ZLKQNKH55ASKFHN42W3PCZYWJS6VREA, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.838078923 (321h21m43.641543517s) [noise=true , address=77.90.31.12:4617 , piece_id=D2P2E5AH3QQRZD2MKT2UOUG7Y4QOARXSFU7MPBVQF2VFCMMYERCQ, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.839295036 (321h21m43.642759636s) [noise=true , address=77.90.31.17:4777 , piece_id=WCHVJ5MDVIGSQWE4VYSG2BRCIYFKGWBAZCQGDDVEEIO4ZIK4MGCQ, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.839737614 (321h21m43.643202212s) [noise=true , address=45.91.116.7:4028 , piece_id=4DJWSDSQ45GSWCZH6H6OVGFWLAXRH27QI4CQAUKJG7Y45KDOV7SA, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.841802119 (321h21m43.645266713s) [noise=true , address=45.91.116.13:4162 , piece_id=RLDKWRPWM3WX6O42BTHBL75LSBSPRBEVDYEHXDBUQ4IN3X5P3OTQ, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.843215985 (321h21m43.646680582s) [noise=true , address=145.79.227.44:4225 , piece_id=VMXBMMZYIF77KFOLGSUAQWXPLACW2MK32DA7VWWXIYZ6J5RG4GCQ, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.843918826 (321h21m43.647383423s) [noise=true , address=145.79.227.43:4171 , piece_id=BTVHWDDV4636C3KSKHJRQI3L7RSO5K46S7EBSOWIZJWOIGFKPY6Q, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.847034063 (321h21m43.650498653s) [noise=true , address=45.91.116.9:4284 , piece_id=TTL5JXNQ7KOIFBQU7HQOGWCDNSWG5ZFIOXEZI22WLZ326H7YO25A, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.849936736 (321h21m43.653401326s) [noise=true , address=45.91.116.5:4316 , piece_id=4V7EM5VI4OHF7CXLPK6EXHP2QUVYT42PK3BFT6243VZ42OBEYTKA, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:55.852057350 (321h21m43.655521951s) [noise=true , address=77.90.31.16:4763 , piece_id=G4RVFEBK4O4QTYBB5AMTL2V6UIF5SCWSCIASBTRDWXUJPINBAK6A, seg_pos={0 0}, upload=282798]: Uploading piece...
Oct 6 22:51:56.437181449 (321h21m44.240646041s) [upload=282798]: Next returned nil segment
Oct 6 22:51:56.437194503 (321h21m44.240659095s) [upload=282798]: Waiting for error group managing segments...
Oct 6 22:51:56.621774513 (321h21m44.425239091s) [noise=true , address=145.79.227.44:4225 , piece_id=VMXBMMZYIF77KFOLGSUAQWXPLACW2MK32DA7VWWXIYZ6J5RG4GCQ, seg_pos={0 0}, upload=282798]: Done uploading piece. err: <nil>
Oct 6 22:51:56.621931773 (321h21m44.425396351s) [noise=true , address=145.79.227.22:4519 , piece_id=YZRNLOAN5YBX734URL2EB2YCLXYVXATCHZKLGWCXOXX2QU63XRUQ, seg_pos={0 0}, upload=282798]: Done uploading piece. err: <nil>
Oct 6 22:51:56.632506254 (321h21m44.435970833s) [noise=true , address=145.79.227.31:4802 , piece_id=YPNDUTRTU7IYGARW2SXIUBEG5SOGWEWRZ4HC4YHD7BQPEC7JZ4UQ, seg_pos={0 0}, upload=282798]: Done uploading piece. err: <nil>
Oct 6 22:51:56.638511638 (321h21m44.441976227s) [noise=true , address=145.79.227.43:4171 , piece_id=BTVHWDDV4636C3KSKHJRQI3L7RSO5K46S7EBSOWIZJWOIGFKPY6Q, seg_pos={0 0}, upload=282798]: Done uploading piece. err: <nil>
Oct 6 22:51:56.645212964 (321h21m44.448677548s) [noise=true , address=145.79.227.30:4792 , piece_id=D3344KOSAB34R26FAEZYQMTHCCNOFJKQOTV46PGTDJEAKCZDGFIA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: <nil>
Oct 6 22:51:57.099414812 (321h21m44.902879399s) [noise=true , address=64.49.13.155:4914 , piece_id=YB4BAA33YQTC4E6ZOGASTE7MDR46ZUJVNXK2VEMTPCZKIILSNNQA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: <nil>
Oct 6 22:52:55.433094081 (321h22m43.236558673s) [noise=true , address=45.91.116.13:4162 , piece_id=RLDKWRPWM3WX6O42BTHBL75LSBSPRBEVDYEHXDBUQ4IN3X5P3OTQ, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:1geUSD7qjW2yuMayTJ1Zy1KTYVC19adZ9nUWMRtmg2TktJSsm4, address:45.91.116.13:4162): protocol: rpc: dial tcp 45.91.116.13:4162: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433274540 (321h22m43.236739125s) [noise=true , address=77.90.31.16:4763 , piece_id=G4RVFEBK4O4QTYBB5AMTL2V6UIF5SCWSCIASBTRDWXUJPINBAK6A, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:12Bn4bKB4U5JidFqnXuPTncmNPGi72tgbL5YmYBRbbJ6ckqo35d, address:77.90.31.16:4763): protocol: rpc: dial tcp 77.90.31.16:4763: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433441208 (321h22m43.236905798s) [noise=true , address=45.91.116.5:4316 , piece_id=4V7EM5VI4OHF7CXLPK6EXHP2QUVYT42PK3BFT6243VZ42OBEYTKA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:1viKEvjPFGZ9E6E9DFPNXZWYE6SYoh4gzxNYqGGK6Mt2q9YciQ, address:45.91.116.5:4316): protocol: read tcp 139.59.144.22:44382->45.91.116.5:4316: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433509768 (321h22m43.23697436s) [noise=true , address=77.90.31.18:4812 , piece_id=BZVDTBRIDDP4PYS553QMYDGGC442A5B4PGK7LBMJQ4RGMFIWQS5Q, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:128qNJJSSwMBPvN8wmTcUeMWAJxVmpjWjwA6WTKxBNCmT2FdFJn, address:77.90.31.18:4812): protocol: read tcp 139.59.144.22:49462->77.90.31.18:4812: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433558250 (321h22m43.237022839s) [noise=true , address=45.91.116.6:4986 , piece_id=M2HAEBACWUTJYNE5PVV7EZVH5KVERGUFCAEU5FODBHLSHOY5E2EA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:12t1cuEr8MRLBbJ4GNSdYBxdhNQ6xDbTr8PHoEuVhMqh1ksgWon, address:45.91.116.6:4986): protocol: read tcp 139.59.144.22:45000->45.91.116.6:4986: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433670048 (321h22m43.237134631s) [noise=true , address=77.90.31.12:4617 , piece_id=D2P2E5AH3QQRZD2MKT2UOUG7Y4QOARXSFU7MPBVQF2VFCMMYERCQ, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:47744->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433755331 (321h22m43.237219914s) [noise=true , address=45.91.116.8:4052 , piece_id=JJ472TZ53QOWOCOC45SNCEWQE76NJO4LM4NPB6TFK2FQWVKACB4A, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:1HHEiq4G4TCaHFJAN3DhL8SwXAFqcwRgYteSHmr1gNe2ThcmnS, address:45.91.116.8:4052): protocol: read tcp 139.59.144.22:33770->45.91.116.8:4052: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433781714 (321h22m43.237246302s) [noise=true , address=45.91.116.4:4962 , piece_id=QPJBUB5IFOCI2WC2QHHNHVGE47U5BNTE2Y7CUKM7EDLOGAAAG4RA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:115Q9FLfn9BcC5kHcdAWa7yn9myRVusdj12Qcy7nvLr1kUXdvF, address:45.91.116.4:4962): protocol: read tcp 139.59.144.22:40746->45.91.116.4:4962: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.433897621 (321h22m43.237362213s) [noise=true , address=45.91.116.11:4147 , piece_id=6U24K2NHSU2KJ4AH3YSM7ZLKQNKH55ASKFHN42W3PCZYWJS6VREA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:12YnWetw1NereHQi7cwHCnmu1LDjRFNpB2MQHSWQrjQdN1CZbMw, address:45.91.116.11:4147): protocol: read tcp 139.59.144.22:49742->45.91.116.11:4147: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.434092748 (321h22m43.237557339s) [noise=true , address=77.90.31.17:4777 , piece_id=WCHVJ5MDVIGSQWE4VYSG2BRCIYFKGWBAZCQGDDVEEIO4ZIK4MGCQ, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:1DjhLCRQRkNECPQg8HG8kZcBn8jJmBZpNhdywxEK7ZUn1v19wM, address:77.90.31.17:4777): protocol: read tcp 139.59.144.22:41604->77.90.31.17:4777: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.434160736 (321h22m43.237625314s) [noise=true , address=45.91.116.9:4284 , piece_id=TTL5JXNQ7KOIFBQU7HQOGWCDNSWG5ZFIOXEZI22WLZ326H7YO25A, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:127CKUmRzR3215HNASZzQmbiy6shYsv98XjmQ2uGu4NjA37KKXx, address:45.91.116.9:4284): protocol: read tcp 139.59.144.22:55276->45.91.116.9:4284: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.434178177 (321h22m43.237642775s) [noise=true , address=45.91.116.7:4028 , piece_id=4DJWSDSQ45GSWCZH6H6OVGFWLAXRH27QI4CQAUKJG7Y45KDOV7SA, seg_pos={0 0}, upload=282798]: Done uploading piece. err: ecclient: upload failed (node:1g3sbXJir88dVbuJCShqFZ5KH2RJhq1kSqfVx6wN7ZkQtucnrB, address:45.91.116.7:4028): protocol: write tcp 139.59.144.22:41728->45.91.116.7:4028: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:55.434211421 (321h22m43.237676012s) [seg_pos={0 0}, upload=282798]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil>
Oct 6 22:52:55.542478957 (321h22m43.345943538s) [upload=282798]: Done uploading segments...
Oct 6 22:52:55.542508398 (321h22m43.34597299s) [upload=282798]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6)
Duration: 1m0.110890534s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.001936791s
FAIL (2025-10-06 22:52:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001991246s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.113609321s
FAIL (2025-10-06 22:52:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
3.266647152 (321h21m51.070111746s) [upload=282801]: Starting upload
Oct 6 22:52:03.266773242 (321h21m51.070237837s) [upload=282801]: Done starting upload
Oct 6 22:52:03.266937035 (321h21m51.070401633s) [upload=282801]: Uploading segments...
Oct 6 22:52:03.267176514 (321h21m51.070641227s) [upload=282801]: Got next segment. Inline: false
Oct 6 22:52:03.268089244 (321h21m51.071553852s) [upload=282801]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:52:03.268111522 (321h21m51.071576123s) [upload=282801]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:52:03.430037228 (321h21m51.233501827s) [seg_pos={0 0}, upload=282801]: Begin upload segment...
Oct 6 22:52:03.430113862 (321h21m51.233578456s) [seg_pos={0 0}, upload=282801]: Done begin upload segment.
Oct 6 22:52:03.430191012 (321h21m51.233655604s) [noise=true , address=145.79.227.5:4223 , piece_id=IZPHPGP4JPK6CYPXZB3O7AEZDR4CHGOSWHK25N45WVRXAWLH4IGQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.430232477 (321h21m51.233697077s) [noise=true , address=145.79.227.3:4151 , piece_id=3FOVZXVEB3253MLNM4XXQXB2MVTOKK3XJZR6H55SDJ4IEUBLLHDQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.430824279 (321h21m51.234288885s) [noise=true , address=77.90.31.17:4770 , piece_id=GFMQGGGYDEUAJNZSMFJHTP5PXEC537E5AYUCRSMNQ674WVZY5TGA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.434338118 (321h21m51.23780271s) [noise=true , address=77.90.31.19:4841 , piece_id=ALRINMNMX7CYVWPIVSVRDPLHIHGZ2VUIGZLRGI5HTRPA5IEDMEAQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.435142913 (321h21m51.238607514s) [noise=true , address=45.91.116.9:4262 , piece_id=W4OXKGKAE35KCKDSNQL3S75QZPIONV5GLOVACWIC3M37ME5RBAAQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.435954054 (321h21m51.239418669s) [noise=true , address=145.79.227.14:4839 , piece_id=SVISSTD52AGG7E7JKNW5HV4ZAIHMQ3CD5PKAGY2O4AYK4PA2PWDQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.436658415 (321h21m51.240123013s) [noise=true , address=145.79.227.17:4969 , piece_id=PAOB6PTNPO7SJP327WRVCJ2STRQQMIJEVLSCGWCAFRXRLRZAV44A, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.437196793 (321h21m51.240661376s) [noise=true , address=77.90.31.11:4553 , piece_id=MGSQPY7RJXHKO7UVWEHGF7ND26PWKEXSSEGXDTH2C3ESIKD436MA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.437784701 (321h21m51.241249295s) [noise=true , address=77.90.31.10:4518 , piece_id=3MSB7KIKPCAFCHUI3UOCYFAEYG7W4I2A4I7W7MAOX6YXHVCKURUA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.438473814 (321h21m51.24193841s) [noise=true , address=45.91.116.12:4227 , piece_id=FLFWRGBERDRKWNZ4V4FXYCZOIPQULHGE3NYOSR2KL3TDPPC6Z4NA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.446137710 (321h21m51.249602322s) [noise=true , address=145.79.227.5:4222 , piece_id=QSJ4JXWZ3BR5H7BXWWJTDZP2BL667L67VEPD3HJMCXAULRYLL6TA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.448705223 (321h21m51.252169836s) [noise=true , address=145.79.227.12:4767 , piece_id=6IPOWLQUCRC2EZOV24BTJV3UQ2W47MW7YFCIK4W3VAYVVFI44MFQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.453854771 (321h21m51.257319379s) [noise=true , address=145.79.227.16:4910 , piece_id=HOCU23QR3XDR7Z5CXAY7J6E3D7DI4XMG5TDJCPVLPXIYZFIFFFQA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.462270326 (321h21m51.265734926s) [noise=true , address=145.79.227.18:4006 , piece_id=46DN34BCNPULTMOKSC2ECHPAZIB53LXIPLM2DWOKL56WJEWCEL4A, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.464373362 (321h21m51.267837968s) [noise=true , address=45.91.116.11:4119 , piece_id=2AYCBJSFSSYYRGFEMR5UWT3OLIEEWXEZVTIUU4N6OUWRSVPIM7YQ, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:03.465236516 (321h21m51.26870112s) [noise=true , address=145.79.227.3:4150 , piece_id=4LD34HU6CUUC67PRQ2ZA7SC2DCAVMKKAXZECKTEB6U4D7VGJOYHA, seg_pos={0 0}, upload=282801]: Uploading piece...
Oct 6 22:52:04.132660648 (321h21m51.936125282s) [upload=282801]: Next returned nil segment
Oct 6 22:52:04.132711626 (321h21m51.936176233s) [upload=282801]: Waiting for error group managing segments...
Oct 6 22:52:04.268577053 (321h21m52.072041634s) [noise=true , address=145.79.227.14:4839 , piece_id=SVISSTD52AGG7E7JKNW5HV4ZAIHMQ3CD5PKAGY2O4AYK4PA2PWDQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.269076811 (321h21m52.072541395s) [noise=true , address=145.79.227.3:4151 , piece_id=3FOVZXVEB3253MLNM4XXQXB2MVTOKK3XJZR6H55SDJ4IEUBLLHDQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.269136691 (321h21m52.072601272s) [noise=true , address=145.79.227.12:4767 , piece_id=6IPOWLQUCRC2EZOV24BTJV3UQ2W47MW7YFCIK4W3VAYVVFI44MFQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.276452785 (321h21m52.079917379s) [noise=true , address=145.79.227.5:4222 , piece_id=QSJ4JXWZ3BR5H7BXWWJTDZP2BL667L67VEPD3HJMCXAULRYLL6TA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.279150664 (321h21m52.082615249s) [noise=true , address=145.79.227.17:4969 , piece_id=PAOB6PTNPO7SJP327WRVCJ2STRQQMIJEVLSCGWCAFRXRLRZAV44A, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.282533290 (321h21m52.085997871s) [noise=true , address=145.79.227.5:4223 , piece_id=IZPHPGP4JPK6CYPXZB3O7AEZDR4CHGOSWHK25N45WVRXAWLH4IGQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.286605407 (321h21m52.090069991s) [noise=true , address=145.79.227.16:4910 , piece_id=HOCU23QR3XDR7Z5CXAY7J6E3D7DI4XMG5TDJCPVLPXIYZFIFFFQA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.291130118 (321h21m52.094594711s) [noise=true , address=145.79.227.18:4006 , piece_id=46DN34BCNPULTMOKSC2ECHPAZIB53LXIPLM2DWOKL56WJEWCEL4A, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:04.296637658 (321h21m52.100102253s) [noise=true , address=145.79.227.3:4150 , piece_id=4LD34HU6CUUC67PRQ2ZA7SC2DCAVMKKAXZECKTEB6U4D7VGJOYHA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: <nil>
Oct 6 22:52:32.905984135 (321h22m20.709448735s) [noise=true , address=45.91.116.9:4262 , piece_id=W4OXKGKAE35KCKDSNQL3S75QZPIONV5GLOVACWIC3M37ME5RBAAQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 139.59.144.22:48884->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906132393 (321h22m20.709596977s) [noise=true , address=77.90.31.17:4770 , piece_id=GFMQGGGYDEUAJNZSMFJHTP5PXEC537E5AYUCRSMNQ674WVZY5TGA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906293048 (321h22m20.70975764s) [noise=true , address=77.90.31.10:4518 , piece_id=3MSB7KIKPCAFCHUI3UOCYFAEYG7W4I2A4I7W7MAOX6YXHVCKURUA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: read tcp 139.59.144.22:59290->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906302895 (321h22m20.709767512s) [noise=true , address=77.90.31.11:4553 , piece_id=MGSQPY7RJXHKO7UVWEHGF7ND26PWKEXSSEGXDTH2C3ESIKD436MA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 139.59.144.22:47872->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906403245 (321h22m20.709867837s) [noise=true , address=45.91.116.11:4119 , piece_id=2AYCBJSFSSYYRGFEMR5UWT3OLIEEWXEZVTIUU4N6OUWRSVPIM7YQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:49046->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906429852 (321h22m20.709894447s) [noise=true , address=77.90.31.19:4841 , piece_id=ALRINMNMX7CYVWPIVSVRDPLHIHGZ2VUIGZLRGI5HTRPA5IEDMEAQ, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:37584->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906437501 (321h22m20.709902093s) [noise=true , address=45.91.116.12:4227 , piece_id=FLFWRGBERDRKWNZ4V4FXYCZOIPQULHGE3NYOSR2KL3TDPPC6Z4NA, seg_pos={0 0}, upload=282801]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 139.59.144.22:37362->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:52:32.906459259 (321h22m20.709923851s) [seg_pos={0 0}, upload=282801]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Oct 6 22:52:33.017661298 (321h22m20.821125895s) [upload=282801]: Done uploading segments...
Oct 6 22:52:33.017697522 (321h22m20.821162119s) [upload=282801]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 30.113638369s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.680351435s
FAIL (2025-10-06 22:49:41 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.680378507s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
3.377880802s
FAIL (2025-10-06 22:48:45 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.377919629s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
2.822191157s
FAIL (2025-10-06 22:48:20 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.82222591s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.992311566s
FAIL (2025-10-06 22:47:44 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.992347608s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
3.409484135s
FAIL (2025-10-06 22:47:14 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.40951286s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.574960185s
FAIL (2025-10-06 22:47:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.57506569s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
2.605683618s
FAIL (2025-10-06 22:46:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.605723148s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.797792922s
FAIL (2025-10-06 22:46:01 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.79786633s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
2.638055937s
FAIL (2025-10-06 22:45:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.638087781s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.771946692s
FAIL (2025-10-06 22:45:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.771994681s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
2.80139026s
FAIL (2025-10-06 22:44:55 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.801415374s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
2.58332412s
FAIL (2025-10-06 22:44:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.583360658s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
3.02221964s
FAIL (2025-10-06 22:43:41 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.022244236s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
3.159381265s
FAIL (2025-10-06 22:42:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 3.159414552s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
2.978725817s
FAIL (2025-10-06 22:42:34 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.978824908s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
2.583909066s
FAIL (2025-10-06 22:42:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.583937332s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
3.031623617s
FAIL (2025-10-06 22:41:57 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.031645088s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
2.748040793s
FAIL (2025-10-06 22:41:12 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.748056186s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
2.642523097s
FAIL (2025-10-06 22:41:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.642554646s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
2.760880127s
FAIL (2025-10-06 22:40:22 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.760961295s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
2.668858877s
FAIL (2025-10-06 22:40:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.668892784s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
3.72197377s
FAIL (2025-10-06 22:40:08 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.722006053s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
2.622651489s
FAIL (2025-10-06 22:39:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.622683993s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
2.947481655s
FAIL (2025-10-06 22:39:26 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.947506425s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
2.563854399s
FAIL (2025-10-06 22:38:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.563885027s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
2.857320678s
FAIL (2025-10-06 22:38:29 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.857341637s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
2.572256154s
FAIL (2025-10-06 22:37:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.57231739s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
3.010640226s
FAIL (2025-10-06 22:37:51 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.010667451s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
3.110249013s
FAIL (2025-10-06 22:37:09 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.110271988s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
2.469645077s
FAIL (2025-10-06 22:37:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.469681795s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
2.72830304s
FAIL (2025-10-06 22:36:07 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.728328138s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
2.688912378s
FAIL (2025-10-06 22:35:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.689080055s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
2.92775488s
FAIL (2025-10-06 22:35:14 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.927784836s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
3.195625798s
FAIL (2025-10-06 22:35:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 3.195663668s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
2.594979864s
FAIL (2025-10-06 22:34:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.595009087s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
2.845911134s
FAIL (2025-10-06 22:34:04 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.846023778s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
2.667031781s
FAIL (2025-10-06 22:33:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.667140178s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
2.86989874s
FAIL (2025-10-06 22:33:28 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.869928131s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
2.574241827s
FAIL (2025-10-06 22:33:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.574276535s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
2.995497577s
FAIL (2025-10-06 22:32:46 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.995558963s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
2.801446399s
FAIL (2025-10-06 22:32:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.801686459s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
3.052662546s
FAIL (2025-10-06 22:31:43 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.052694155s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
2.919345319s
FAIL (2025-10-06 22:31:21 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.919819185s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
2.590493472s
FAIL (2025-10-06 22:31:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload
Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload
Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments...
Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false
Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment...
Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment.
Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece...
Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment
Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments...
Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil>
Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments...
Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.590530345s
core
uplink
us-select-1
|