ssh failed, ssh can't send tcp package to PC

OutofSoul — Wed, 12 Apr 2023 08:43:23 GMT

environment

soc: s32g399ardb3
net device: aux0sl (pfeslave)

phenomenon description

I ssh the PC(ubuntu) from embedded Linux. But after authentication, it can't start shell.

Through the `tcpdump` log, i found the tcp data pkgs (include ack pkgs) can't be sent to PC, after authenticaion.

But before authentication, the data pkgs can be sent to PC from s32g, the reverse is also true.

log

net device infomation

# after boot kernel, aux0sl isn't up, i have to up it manually root@s32g399ardb3:~# ifconfig aux0sl 192.168.123.120 root@s32g399ardb3:~# ifconfig aux0sl: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.123.120 netmask 255.255.255.0 broadcast 192.168.123.255 inet6 fe80::204:9fff:febe:ff80 prefixlen 64 scopeid 0x20<link> ether 00:04:9f:be:ff:80 txqueuelen 1000 (Ethernet) RX packets 7811 bytes 524882 (512.5 KiB) RX errors 0 dropped 7376 overruns 0 frame 0 TX packets 19 bytes 1532 (1.4 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device memory 0x46000000-46ffffff

the ssh log

tcpdump log

root@s32g399ardb3:~# tcpdump -i aux0sl -n host 192.168.123.111 & root@s32g399ardb3:~# ssh cg@192.168.123.111 17:26:35.072053 ARP, Request who-has 192.168.123.111 tell 192.168.123.1, length 46 17:26:39.961413 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [S], seq 326276919, win 64240, options [mss 1460,sackOK,TS val 3645630678 ecr 0,nop,wscale 7], length 0 17:26:39.962103 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [S.], seq 1923967336, ack 326276920, win 65160, options [mss 1460,sackOK,TS val 3682564497 ecr 3645630678,nop,wscale 7], length 0 17:26:39.962180 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [.], ack 1, win 502, options [nop,nop,TS val 3645630679 ecr 3682564497], length 0 17:26:39.963144 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1:22, ack 1, win 502, options [nop,nop,TS val 3645630680 ecr 3682564497], length 21 17:26:39.963537 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [.], ack 22, win 509, options [nop,nop,TS val 3682564498 ecr 3645630680], length 0 17:26:39.965709 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [P.], seq 1:42, ack 22, win 509, options [nop,nop,TS val 3682564500 ecr 3645630680], length 41 17:26:39.965731 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [.], ack 42, win 502, options [nop,nop,TS val 3645630682 ecr 3682564500], length 0 17:26:39.966446 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [P.], seq 42:1122, ack 22, win 509, options [nop,nop,TS val 3682564501 ecr 3645630682], length 1080 17:26:39.966465 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [.], ack 1122, win 501, options [nop,nop,TS val 3645630683 ecr 3682564501], length 0 17:26:39.967380 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [.], seq 22:1470, ack 1122, win 501, options [nop,nop,TS val 3645630684 ecr 3682564501], length 1448 17:26:39.967398 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1470:1534, ack 1122, win 501, options [nop,nop,TS val 3645630684 ecr 3682564501], length 64 17:26:39.968045 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [.], ack 1534, win 499, options [nop,nop,TS val 3682564503 ecr 3645630684], length 0 17:26:39.983024 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1534:1582, ack 1122, win 501, options [nop,nop,TS val 3645630700 ecr 3682564503], length 48 17:26:39.983527 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [.], ack 1582, win 501, options [nop,nop,TS val 3682564518 ecr 3645630700], length 0 17:26:39.986271 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [P.], seq 1122:1574, ack 1582, win 501, options [nop,nop,TS val 3682564521 ecr 3645630700], length 452 17:26:39.986288 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [.], ack 1574, win 501, options [nop,nop,TS val 3645630703 ecr 3682564521], length 0 17:26:40.003957 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1582:1598, ack 1574, win 501, options [nop,nop,TS val 3645630720 ecr 3682564521], length 16 17:26:40.004534 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [.], ack 1598, win 501, options [nop,nop,TS val 3682564539 ecr 3645630720], length 0 17:26:40.004556 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1598:1642, ack 1574, win 501, options [nop,nop,TS val 3645630721 ecr 3682564539], length 44 17:26:40.004886 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [.], ack 1642, win 501, options [nop,nop,TS val 3682564540 ecr 3645630721], length 0 17:26:40.004977 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [P.], seq 1574:1618, ack 1642, win 501, options [nop,nop,TS val 3682564540 ecr 3645630721], length 44 17:26:40.004994 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [.], ack 1618, win 501, options [nop,nop,TS val 3645630722 ecr 3682564540], length 0 17:26:40.005062 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1642:1702, ack 1618, win 501, options [nop,nop,TS val 3645630722 ecr 3682564540], length 60 17:26:40.005346 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [.], ack 1702, win 501, options [nop,nop,TS val 3682564540 ecr 3645630722], length 0 17:26:40.005712 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [P.], seq 1618:1670, ack 1702, win 501, options [nop,nop,TS val 3682564540 ecr 3645630722], length 52 17:26:40.052963 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [.], ack 1670, win 501, options [nop,nop,TS val 3645630770 ecr 3682564540], length 0 cg@192.168.123.111's password: 17:26:42.785073 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1786:1870, ack 1722, win 501, options [nop,nop,TS val 3645633502 ecr 3682564888], length 84 17:26:42.788922 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [P.], seq 1722:1750, ack 1870, win 501, options [nop,nop,TS val 3682567322 ecr 3645633502], length 28 17:26:42.788944 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [.], ack 1750, win 501, options [nop,nop,TS val 3645633505 ecr 3682567322], length 0 17:26:42.837026 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1870:1982, ack 1750, win 501, options [nop,nop,TS val 3645633554 ecr 3682567322], length 112 17:26:42.871872 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [P.], seq 1750:2250, ack 1982, win 501, options [nop,nop,TS val 3682567405 ecr 3645633554], length 500 17:26:42.871893 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [.], ack 2250, win 501, options [nop,nop,TS val 3645633588 ecr 3682567405], length 0 17:26:42.872232 IP 192.168.123.111.22 > 192.168.123.120.34874: Flags [P.], seq 2250:2294, ack 1982, win 501, options [nop,nop,TS val 3682567405 ecr 3645633588], length 44 # after here, data pkgs can't be sent out 17:26:42.872251 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [.], ack 2294, win 501, options [nop,nop,TS val 3645633589 ecr 3682567405], length 0 17:26:42.872403 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1982:2358, ack 2294, win 501, options [nop,nop,TS val 3645633589 ecr 3682567405], length 376 17:26:43.080966 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1982:2358, ack 2294, win 501, options [nop,nop,TS val 3645633798 ecr 3682567405], length 376 17:26:43.292969 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1982:2358, ack 2294, win 501, options [nop,nop,TS val 3645634010 ecr 3682567405], length 376 17:26:43.720966 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1982:2358, ack 2294, win 501, options [nop,nop,TS val 3645634438 ecr 3682567405], length 376 17:26:44.584990 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1982:2358, ack 2294, win 501, options [nop,nop,TS val 3645635302 ecr 3682567405], length 376 17:26:46.276975 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1982:2358, ack 2294, win 501, options [nop,nop,TS val 3645636994 ecr 3682567405], length 376 17:26:47.071840 ARP, Request who-has 192.168.123.111 tell 192.168.123.1, length 46 17:26:49.672970 IP 192.168.123.120.34874 > 192.168.123.111.22: Flags [P.], seq 1982:2358, ack 2294, win 501, options [nop,nop,TS val 3645640390 ecr 3682567405], length 376

the successful log

I tired use eth0(gmac) instead of aux0sl(pfeslave), and it can ssh PC successfully.

What causes the difference between these two situations？

root@s32g399ardb3:~# ssh -v cg@192.168.123.111 OpenSSH_8.3p1, OpenSSL 1.1.1k 25 Mar 2021 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 20: Applying options for * debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling debug1: Connecting to 192.168.123.111 [192.168.123.111] port 22. debug1: Connection established. debug1: identity file /home/root/.ssh/id_rsa type -1 debug1: identity file /home/root/.ssh/id_rsa-cert type -1 debug1: identity file /home/root/.ssh/id_dsa type -1 debug1: identity file /home/root/.ssh/id_dsa-cert type -1 debug1: identity file /home/root/.ssh/id_ecdsa type -1 debug1: identity file /home/root/.ssh/id_ecdsa-cert type -1 debug1: identity file /home/root/.ssh/id_ecdsa_sk type -1 debug1: identity file /home/root/.ssh/id_ecdsa_sk-cert type -1 debug1: identity file /home/root/.ssh/id_ed25519 type -1 debug1: identity file /home/root/.ssh/id_ed25519-cert type -1 debug1: identity file /home/root/.ssh/id_ed25519_sk type -1 debug1: identity file /home/root/.ssh/id_ed25519_sk-cert type -1 debug1: identity file /home/root/.ssh/id_xmss type -1 debug1: identity file /home/root/.ssh/id_xmss-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_8.3 debug1: Remote protocol version 2.0, remote software version OpenSSH_7.6p1 Ubuntu-4ubuntu0.7 debug1: match: OpenSSH_7.6p1 Ubuntu-4ubuntu0.7 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002 debug1: Authenticating to 192.168.123.111:22 as 'cg' debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: algorithm: curve25519-sha256 debug1: kex: host key algorithm: ecdsa-sha2-nistp256 debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug1: Server host key: ecdsa-sha2-nistp256 SHA256:EZwQIrHaRi521tPos2AwwtISifFnRSWkJUMaa4fMxUU debug1: Host '192.168.123.111' is known and matches the ECDSA host key. debug1: Found key in /home/root/.ssh/known_hosts:2 debug1: rekey out after 134217728 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: rekey in after 134217728 blocks debug1: Will attempt key: /home/root/.ssh/id_rsa debug1: Will attempt key: /home/root/.ssh/id_dsa debug1: Will attempt key: /home/root/.ssh/id_ecdsa debug1: Will attempt key: /home/root/.ssh/id_ecdsa_sk debug1: Will attempt key: /home/root/.ssh/id_ed25519 debug1: Will attempt key: /home/root/.ssh/id_ed25519_sk debug1: Will attempt key: /home/root/.ssh/id_xmss debug1: SSH2_MSG_EXT_INFO received debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521> debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,password debug1: Next authentication method: publickey debug1: Trying private key: /home/root/.ssh/id_rsa debug1: Trying private key: /home/root/.ssh/id_dsa debug1: Trying private key: /home/root/.ssh/id_ecdsa debug1: Trying private key: /home/root/.ssh/id_ecdsa_sk debug1: Trying private key: /home/root/.ssh/id_ed25519 debug1: Trying private key: /home/root/.ssh/id_ed25519_sk debug1: Trying private key: /home/root/.ssh/id_xmss debug1: Next authentication method: password cg@192.168.123.111's password: debug1: Authentication succeeded (password). Authenticated to 192.168.123.111 ([192.168.123.111]:22). debug1: channel 0: new [client-session] debug1: Requesting no-more-sessions@openssh.com debug1: Entering interactive session. debug1: pledge: exec debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0 debug1: X11 forwarding requested but DISPLAY not set Welcome to Kylin V10 SP1 (GNU/Linux 5.4.0-144-generic x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage 56 updates can be applied immediately. 21 of these updates are standard security updates. To see these additional updates run: apt list --upgradable Last login: Tue Apr 11 16:54:41 2023 from 192.168.123.120 cg@laptop:~$

Re: ssh failed, ssh can't send tcp package to PC

MayanksPatel — Fri, 14 Apr 2023 11:58:55 GMT

Hi @OutofSoul,

Thank you for reaching out to NXP.

Can you please share tcp/ip stack and PFE Firmware version and Linux BSP version which you are using?

Can you please describe more about the setup?

Thanks,

Mayank s Patel

Re: ssh failed, ssh can't send tcp package to PC

OutofSoul — Mon, 24 Apr 2023 07:53:39 GMT

Hello, sorry for taking so long to reply.

Because NXP's techinical support personnel provided a solution brefore, i haven't followed the forum since then.

This is an issue with the TX queue of PFE.

Re: ssh failed, ssh can't send tcp package to PC

MayanksPatel — Mon, 24 Apr 2023 09:16:28 GMT

Hi @OutofSoul,

Thanks to inform us. We are closing this issue.

Thanks,

Mayank s Patel

topic Re: ssh failed, ssh can't send tcp package to PC in S32G

ssh failed, ssh can't send tcp package to PC

environment

phenomenon description

log

the successful log

Re: ssh failed, ssh can't send tcp package to PC

Re: ssh failed, ssh can't send tcp package to PC

Re: ssh failed, ssh can't send tcp package to PC