網路通信基礎重難點解析 06 ：send 和 recv 函數在阻塞和非阻塞模式下的行為
send 和 recv 函數在阻塞和非阻塞模式下的行為
send 和 recv 函數其實名不符實。
send 函數本質上並不是往網路上發送數據，而是將應用層發送緩衝區的數據拷貝到內核緩衝區（下文為了敘述方便，我們以「網卡緩衝區」代指）中去，至於什麼時候數據會從網卡緩衝區中真正地發到網路中去要根據 TCP/IP 協議棧的行為來確定，這種行為涉及到一個叫 nagel 演算法和 TCP_NODELAY 的 socket 選項，我們將在《nagle演算法與 TCP_NODELAY》章節詳細介紹。
recv 函數本質上也並不是從網路上收取數據，而只是將內核緩衝區中的數據拷貝到應用程序的緩衝區中，當然拷貝完成以後會將內核緩衝區中該部分數據移除。
可以用下面一張圖來描述上述事實：
通過上圖我們知道，不同的程序進行網路通信時，發送的一方會將內核緩衝區的數據通過網路傳輸給接收方的內核緩衝區。在應用程序 A 與 應用程序 B 建立了 TCP 連接之後，假設應用程序 A 不斷調用 send 函數，這樣數據會不斷拷貝至對應的內核緩衝區中，如果 B 那一端一直不調用 recv 函數，那麼 B 的內核緩衝區被填滿以後，A 的內核緩衝區也會被填滿，此時 A 繼續調用 send 函數會是什麼結果呢？ 具體的結果取決於該 socket 是否是阻塞模式。我們這裡先給出結論：
- 當 socket 是阻塞模式的，繼續調用 send/recv 函數會導致程序阻塞在 send/recv 調用處。
- 當 socket 是非阻塞模式，繼續調用 send/recv 函數，send/recv 函數不會阻塞程序執行流，而是會立即出錯返回，我們會得到一個相關的錯誤碼，Linux 平台上該錯誤碼為 EWOULDBLOCK 或 EAGAIN（這兩個錯誤碼值相同），Windows 平台上錯誤碼為 WSAEWOULDBLOCK。
我們實際來編寫一下代碼來驗證一下以上說的兩種情況。
socket 阻塞模式下的 send 行為
服務端代碼（blocking_server.cpp）如下：
/**
* 驗證阻塞模式下send函數的行為，server端
* zhangyl 2018.12.17
*/
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <iostream>
#include <string.h>
int main(int argc, char* argv[])
{
//1.創建一個偵聽socket
int listenfd = socket(AF_INET, SOCK_STREAM, 0);
if (listenfd == -1)
{
std::cout << "create listen socket error." << std::endl;
return -1;
}
//2.初始化伺服器地址
struct sockaddr_in bindaddr;
bindaddr.sin_family = AF_INET;
bindaddr.sin_addr.s_addr = htonl(INADDR_ANY);
bindaddr.sin_port = htons(3000);
if (bind(listenfd, (struct sockaddr *)&bindaddr, sizeof(bindaddr)) == -1)
{
std::cout << "bind listen socket error." << std::endl;
close(listenfd);
return -1;
}
//3.啟動偵聽
if (listen(listenfd, SOMAXCONN) == -1)
{
std::cout << "listen error." << std::endl;
close(listenfd);
return -1;
}
while (true)
{
struct sockaddr_in clientaddr;
socklen_t clientaddrlen = sizeof(clientaddr);
//4. 接受客戶端連接
int clientfd = accept(listenfd, (struct sockaddr *)&clientaddr, &clientaddrlen);
if (clientfd != -1)
{
//只接受連接，不調用recv收取任何數據
std:: cout << "accept a client connection." << std::endl;
}
}
//7.關閉偵聽socket
close(listenfd);
return 0;
}
客戶端代碼（blocking_client.cpp）如下：
/**
* 驗證阻塞模式下send函數的行為，client端
* zhangyl 2018.12.17
*/
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <iostream>
#include <string.h>
#define SERVER_ADDRESS "127.0.0.1"
#define SERVER_PORT 3000
#define SEND_DATA "helloworld"
int main(int argc, char* argv[])
{
//1.創建一個socket
int clientfd = socket(AF_INET, SOCK_STREAM, 0);
if (clientfd == -1)
{
std::cout << "create client socket error." << std::endl;
close(clientfd);
return -1;
}
//2.連接伺服器
struct sockaddr_in serveraddr;
serveraddr.sin_family = AF_INET;
serveraddr.sin_addr.s_addr = inet_addr(SERVER_ADDRESS);
serveraddr.sin_port = htons(SERVER_PORT);
if (connect(clientfd, (struct sockaddr *)&serveraddr, sizeof(serveraddr)) == -1)
{
std::cout << "connect socket error." << std::endl;
return -1;
}
//3. 不斷向伺服器發送數據，或者出錯退出
int count = 0;
while (true)
{
int ret = send(clientfd, SEND_DATA, strlen(SEND_DATA), 0);
if (ret != strlen(SEND_DATA))
{
std::cout << "send data error." << std::endl;
break;
}
else
{
count ++;
std::cout << "send data successfully, count = " << count << std::endl;
}
}
//5. 關閉socket
close(clientfd);
return 0;
}
在 shell 中分別編譯這兩個 cpp 文件得到兩個可執行程序 blocking_server 和 blocking_client：
g++ -g -o blocking_server blocking_server.cpp
g++ -g -o blocking_client blocking_client.cpp
我們先啟動 blocking_server，然後用 gdb 啟動 blocking_client，輸入 run 命令讓 blocking_client跑起來，blocking_client 會不斷地向 blocking_server 發送"helloworld"字元串，每次 send 成功後，會將計數器 count 的值列印出來，計數器會不斷增加，程序運行一段時間後，計數器 count 值不再增加且程序不再有輸出。操作過程及輸出結果如下：
blocking_server 端：
[root@localhost testsocket]# ./blocking_server
accept a client connection.
[root@localhost testsocket]# gdb blocking_client
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-100.el7_4.1
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /root/testsocket/blocking_client...done.
(gdb) run
//輸出結果太多，省略部分...
send data successfully, count = 355384
send data successfully, count = 355385
send data successfully, count = 355386
send data successfully, count = 355387
send data successfully, count = 355388
send data successfully, count = 355389
send data successfully, count = 355390
此時程序不再有輸出，說明我們的程序應該「卡在」某個地方，繼續按 Ctrl + C 讓 gdb 中斷下來，輸入 bt 命令查看此時的調用堆棧，我們發現我們的程序確實阻塞在 send 函數調用處：
^C
Program received signal SIGINT, Interrupt.
0x00007ffff72f130d in send () from /lib64/libc.so.6
(gdb) bt
#0 0x00007ffff72f130d in send () from /lib64/libc.so.6
#1 0x0000000000400b46 in main (argc=1, argv=0x7fffffffe598) at blocking_client.cpp:41
(gdb)
上面的示例驗證了如果一端一直發數據，而對端應用層一直不取數據（或收取數據的速度慢於發送速度），則很快兩端的內核緩衝區很快就會被填滿，導致發送端調用 send 函數被阻塞。這裡說的「內核緩衝區」 其實有個專門的名字，即 TCP 窗口。也就是說 socket 阻塞模式下， send 函數在 TCP 窗口太小時的行為是阻塞當前程序執行流（即阻塞 send 函數所在的線程的執行）。
說點題外話，上面的例子，我們每次發送一個「helloworld」（10個位元組），一共發了 355390 次（每次測試的結果略有不同），我們可以粗略地算出 TCP 窗口的大小大約等於 1.7 M左右 （10 * 355390 / 2）。
讓我們再深入一點，我們利用 Linux tcpdump 工具來動態看一下這種情形下 TCP 窗口大小的動態變化。需要注意的是，Linux 下使用 tcpdump 這個命令需要有 root 許可權。
我們開啟三個 shell 窗口，在第一個窗口先啟動 blocking_server 進程，在第二個窗口用 tcpdump 抓經過 TCP 埠 3000 上的數據包：
[root@localhost testsocket]# tcpdump -i any -nn -S tcp port 3000
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
接著在第三個 shell 窗口，啟動 blocking_client。當 blocking_client 進程不再輸出時，我們抓包的結果如下：
[root@localhost testsocket]# tcpdump -i any -nn -S tcp port 3000
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
11:52:35.907381 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [S], seq 1394135076, win 43690, options [mss 65495,sackOK,TS val 78907688 ecr 0,nop,wscale 7], length 0
20:32:21.261484 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [S.], seq 1233000591, ack 1394135077, win 43690, options [mss 65495,sackOK,TS val 78907688 ecr 78907688,nop,wscale 7], length 0
11:52:35.907441 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [.], ack 1233000592, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 0
11:52:35.907615 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394135077:1394135087, ack 1233000592, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 10
11:52:35.907626 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394135087, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 0
11:52:35.907785 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394135087:1394135097, ack 1233000592, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 10
11:52:35.907793 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394135097, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 0
11:52:35.907809 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394135097:1394135107, ack 1233000592, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 10
11:52:35.907814 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394135107, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 0
11:52:35.907839 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394135117:1394135127, ack 1233000592, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 10
11:52:35.907853 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394135127:1394135137, ack 1233000592, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 10
11:52:35.907880 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394135147:1394135157, ack 1233000592, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 10
11:52:35.907896 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394135167, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 0
11:52:35.907920 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394135177:1394135187, ack 1233000592, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 10
11:52:35.907924 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394135187, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 0
11:52:35.907938 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394135197, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 0
11:52:35.923799 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [.], seq 1394135247:1394157135, ack 1233000592, win 342, options [nop,nop,TS val 78907704 ecr 78907688], length 21888
11:52:35.923840 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394157135, win 1365, options [nop,nop,TS val 78907704 ecr 78907688], length 0
11:52:35.923851 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394157135:1394157137, ack 1233000592, win 342, options [nop,nop,TS val 78907704 ecr 78907704], length 2
11:52:35.964013 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394157137, win 1365, options [nop,nop,TS val 78907744 ecr 78907704], length 0
11:52:35.964036 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394157137:1394170597, ack 1233000592, win 342, options [nop,nop,TS val 78907744 ecr 78907744], length 13460
11:52:35.964043 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394170597, win 2388, options [nop,nop,TS val 78907744 ecr 78907744], length 0
11:52:36.081009 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394170597:1394170607, ack 1233000592, win 342, options [nop,nop,TS val 78907861 ecr 78907744], length 10
11:52:36.121161 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394170607, win 2388, options [nop,nop,TS val 78907901 ecr 78907861], length 0
11:52:36.121252 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394170607:1394188957, ack 1233000592, win 342, options [nop,nop,TS val 78907902 ecr 78907901], length 18350
11:52:36.121285 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394188957, win 3411, options [nop,nop,TS val 78907902 ecr 78907902], length 0
11:52:36.243907 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394188957:1394188967, ack 1233000592, win 342, options [nop,nop,TS val 78908024 ecr 78907902], length 10
11:52:36.284003 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394188967, win 3411, options [nop,nop,TS val 78908064 ecr 78908024], length 0
11:52:36.284024 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394188967:1394207257, ack 1233000592, win 342, options [nop,nop,TS val 78908064 ecr 78908064], length 18290
11:52:36.284034 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394207257, win 3635, options [nop,nop,TS val 78908064 ecr 78908064], length 0
11:52:36.409664 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394207257:1394207267, ack 1233000592, win 342, options [nop,nop,TS val 78908190 ecr 78908064], length 10
11:52:36.450133 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394207267, win 3635, options [nop,nop,TS val 78908230 ecr 78908190], length 0
11:52:36.450187 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394207267:1394224477, ack 1233000592, win 342, options [nop,nop,TS val 78908231 ecr 78908230], length 17210
11:52:36.491025 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394224477, win 3635, options [nop,nop,TS val 78908271 ecr 78908231], length 0
11:52:36.569355 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394224477:1394224487, ack 1233000592, win 342, options [nop,nop,TS val 78908350 ecr 78908271], length 10
11:52:36.609958 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394224487, win 3635, options [nop,nop,TS val 78908390 ecr 78908350], length 0
11:52:36.609993 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394224487:1394242657, ack 1233000592, win 342, options [nop,nop,TS val 78908390 ecr 78908390], length 18170
11:52:36.650181 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394242657, win 3635, options [nop,nop,TS val 78908430 ecr 78908390], length 0
11:52:36.731142 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394242657:1394242667, ack 1233000592, win 342, options [nop,nop,TS val 78908511 ecr 78908430], length 10
11:52:36.771958 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394242667, win 3635, options [nop,nop,TS val 78908552 ecr 78908511], length 0
11:52:36.771979 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394242667:1394258437, ack 1233000592, win 342, options [nop,nop,TS val 78908552 ecr 78908552], length 15770
11:52:36.811957 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394258437, win 3580, options [nop,nop,TS val 78908592 ecr 78908552], length 0
11:52:36.892064 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394258437:1394258447, ack 1233000592, win 342, options [nop,nop,TS val 78908672 ecr 78908592], length 10
11:52:36.932039 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394258447, win 3580, options [nop,nop,TS val 78908712 ecr 78908672], length 0
11:52:36.932082 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394258447:1394276017, ack 1233000592, win 342, options [nop,nop,TS val 78908712 ecr 78908712], length 17570
11:52:36.971983 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394276017, win 3499, options [nop,nop,TS val 78908752 ecr 78908712], length 0
11:52:37.056904 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394276017:1394276027, ack 1233000592, win 342, options [nop,nop,TS val 78908837 ecr 78908752], length 10
11:52:37.096966 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394276027, win 3499, options [nop,nop,TS val 78908877 ecr 78908837], length 0
11:52:37.096989 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394276027:1394291677, ack 1233000592, win 342, options [nop,nop,TS val 78908877 ecr 78908877], length 15650
11:52:37.136941 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394291677, win 3425, options [nop,nop,TS val 78908917 ecr 78908877], length 0
11:52:37.223275 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394291677:1394291687, ack 1233000592, win 342, options [nop,nop,TS val 78909004 ecr 78908917], length 10
11:52:37.263951 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394291687, win 3425, options [nop,nop,TS val 78909044 ecr 78909004], length 0
11:52:37.263974 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394291687:1394308477, ack 1233000592, win 342, options [nop,nop,TS val 78909044 ecr 78909044], length 16790
11:52:37.303956 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394308477, win 3347, options [nop,nop,TS val 78909084 ecr 78909044], length 0
11:52:37.383620 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394308477:1394308487, ack 1233000592, win 342, options [nop,nop,TS val 78909164 ecr 78909084], length 10
11:52:37.423926 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394308487, win 3347, options [nop,nop,TS val 78909204 ecr 78909164], length 0
11:52:37.423952 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394308487:1394326297, ack 1233000592, win 342, options [nop,nop,TS val 78909204 ecr 78909204], length 17810
11:52:37.463914 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394326297, win 3266, options [nop,nop,TS val 78909244 ecr 78909204], length 0
11:52:37.545414 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394326297:1394326307, ack 1233000592, win 342, options [nop,nop,TS val 78909326 ecr 78909244], length 10
11:52:37.586052 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394326307, win 3266, options [nop,nop,TS val 78909366 ecr 78909326], length 0
11:52:37.586109 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394326307:1394343097, ack 1233000592, win 342, options [nop,nop,TS val 78909366 ecr 78909366], length 16790
11:52:37.626049 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394343097, win 3188, options [nop,nop,TS val 78909406 ecr 78909366], length 0
11:52:37.707516 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394343097:1394343107, ack 1233000592, win 342, options [nop,nop,TS val 78909488 ecr 78909406], length 10
11:52:37.747870 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394343107, win 3188, options [nop,nop,TS val 78909528 ecr 78909488], length 0
11:52:37.747892 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394343107:1394358877, ack 1233000592, win 342, options [nop,nop,TS val 78909528 ecr 78909528], length 15770
11:52:37.787982 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394358877, win 3114, options [nop,nop,TS val 78909568 ecr 78909528], length 0
11:52:38.068368 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394358877:1394358887, ack 1233000592, win 342, options [nop,nop,TS val 78909849 ecr 78909568], length 10
11:52:38.068434 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394358887, win 3114, options [nop,nop,TS val 78909849 ecr 78909849], length 0
11:52:38.068459 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394358887:1394358897, ack 1233000592, win 342, options [nop,nop,TS val 78909849 ecr 78909849], length 10
11:52:38.068461 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394358897, win 3114, options [nop,nop,TS val 78909849 ecr 78909849], length 0
11:52:38.068466 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394358897:1394358907, ack 1233000592, win 342, options [nop,nop,TS val 78909849 ecr 78909849], length 10
11:52:38.068467 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394358907, win 3114, options [nop,nop,TS val 78909849 ecr 78909849], length 0
11:52:38.068471 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394358907:1394358917, ack 1233000592, win 342, options [nop,nop,TS val 78909849 ecr 78909849], length 10
11:52:38.068472 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394358917, win 3114, options [nop,nop,TS val 78909849 ecr 78909849], length 0
11:52:38.068475 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394358917:1394358927, ack 1233000592, win 342, options [nop,nop,TS val 78909849 ecr 78909849], length 10
11:52:38.068476 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394358927, win 3114, options [nop,nop,TS val 78909849 ecr 78909849], length 0
11:52:38.068480 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394358937, win 3114, options [nop,nop,TS val 78909849 ecr 78909849], length 0
11:52:38.068483 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394358937:1394358947, ack 1233000592, win 342, options [nop,nop,TS val 78909849 ecr 78909849], length 10
11:52:38.068488 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394358947:1394358957, ack 1233000592, win 342, options [nop,nop,TS val 78909849 ecr 78909849], length 10
11:52:38.068491 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394358957:1394358967, ack 1233000592, win 342, options [nop,nop,TS val 78909849 ecr 78909849], length 10
11:52:38.068496 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394358967:1394358977, ack 1233000592, win 342, options [nop,nop,TS val 78909849 ecr 78909849], length 10
11:52:38.108851 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394358977, win 3114, options [nop,nop,TS val 78909889 ecr 78909849], length 0
11:52:38.108910 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394358977:1394376697, ack 1233000592, win 342, options [nop,nop,TS val 78909889 ecr 78909889], length 17720
11:52:38.149011 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394376697, win 3032, options [nop,nop,TS val 78909929 ecr 78909889], length 0
11:52:38.234851 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394376697:1394376707, ack 1233000592, win 342, options [nop,nop,TS val 78910015 ecr 78909929], length 10
11:52:38.274934 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394376707, win 3032, options [nop,nop,TS val 78910055 ecr 78910015], length 0
11:52:38.274949 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394376707:1394392477, ack 1233000592, win 342, options [nop,nop,TS val 78910055 ecr 78910055], length 15770
11:52:38.314919 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394392477, win 2958, options [nop,nop,TS val 78910095 ecr 78910055], length 0
11:52:38.314940 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394392477:1394409277, ack 1233000592, win 342, options [nop,nop,TS val 78910095 ecr 78910095], length 16800
11:52:38.354895 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394409277, win 2889, options [nop,nop,TS val 78910135 ecr 78910095], length 0
11:52:38.354913 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394409277:1394426077, ack 1233000592, win 342, options [nop,nop,TS val 78910135 ecr 78910135], length 16800
11:52:38.394876 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394426077, win 2820, options [nop,nop,TS val 78910175 ecr 78910135], length 0
11:52:38.394890 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394426077:1394442877, ack 1233000592, win 342, options [nop,nop,TS val 78910175 ecr 78910175], length 16800
11:52:38.434909 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394442877, win 2751, options [nop,nop,TS val 78910215 ecr 78910175], length 0
11:52:38.434925 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394442877:1394459677, ack 1233000592, win 342, options [nop,nop,TS val 78910215 ecr 78910215], length 16800
11:52:38.474890 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394459677, win 2683, options [nop,nop,TS val 78910255 ecr 78910215], length 0
11:52:38.474901 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394459677:1394470177, ack 1233000592, win 342, options [nop,nop,TS val 78910255 ecr 78910255], length 10500
11:52:38.514891 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394470177, win 2638, options [nop,nop,TS val 78910295 ecr 78910255], length 0
11:52:38.514908 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394470177:1394476477, ack 1233000592, win 342, options [nop,nop,TS val 78910295 ecr 78910295], length 6300
11:52:38.554921 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394476477, win 2610, options [nop,nop,TS val 78910335 ecr 78910295], length 0
11:52:38.554941 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394476477:1394493277, ack 1233000592, win 342, options [nop,nop,TS val 78910335 ecr 78910335], length 16800
11:52:38.594888 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394493277, win 2541, options [nop,nop,TS val 78910375 ecr 78910335], length 0
11:52:38.594905 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394493277:1394510077, ack 1233000592, win 342, options [nop,nop,TS val 78910375 ecr 78910375], length 16800
11:52:38.634904 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394510077, win 2472, options [nop,nop,TS val 78910415 ecr 78910375], length 0
11:52:38.634922 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394510077:1394527897, ack 1233000592, win 342, options [nop,nop,TS val 78910415 ecr 78910415], length 17820
11:52:38.674852 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394527897, win 2400, options [nop,nop,TS val 78910455 ecr 78910415], length 0
11:52:38.674864 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394527897:1394544637, ack 1233000592, win 342, options [nop,nop,TS val 78910455 ecr 78910455], length 16740
11:52:38.714870 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394544637, win 2331, options [nop,nop,TS val 78910495 ecr 78910455], length 0
11:52:38.714897 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394544637:1394560337, ack 1233000592, win 342, options [nop,nop,TS val 78910495 ecr 78910495], length 15700
11:52:38.754893 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394560337, win 2266, options [nop,nop,TS val 78910535 ecr 78910495], length 0
11:52:38.754925 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394560337:1394560357, ack 1233000592, win 342, options [nop,nop,TS val 78910535 ecr 78910535], length 20
11:52:38.794922 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394560357, win 2266, options [nop,nop,TS val 78910575 ecr 78910535], length 0
11:52:38.794942 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394560357:1394577157, ack 1233000592, win 342, options [nop,nop,TS val 78910575 ecr 78910575], length 16800
11:52:38.834848 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394577157, win 2188, options [nop,nop,TS val 78910615 ecr 78910575], length 0
11:52:38.834868 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394577157:1394595037, ack 1233000592, win 342, options [nop,nop,TS val 78910615 ecr 78910615], length 17880
11:52:38.874858 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394595037, win 2115, options [nop,nop,TS val 78910655 ecr 78910615], length 0
11:52:38.874878 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394595037:1394610757, ack 1233000592, win 342, options [nop,nop,TS val 78910655 ecr 78910655], length 15720
11:52:38.914841 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394610757, win 2050, options [nop,nop,TS val 78910695 ecr 78910655], length 0
11:52:38.914854 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394610757:1394628637, ack 1233000592, win 342, options [nop,nop,TS val 78910695 ecr 78910695], length 17880
11:52:38.954812 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394628637, win 1977, options [nop,nop,TS val 78910735 ecr 78910695], length 0
11:52:38.954841 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394628637:1394645437, ack 1233000592, win 342, options [nop,nop,TS val 78910735 ecr 78910735], length 16800
11:52:38.994885 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394645437, win 1908, options [nop,nop,TS val 78910775 ecr 78910735], length 0
11:52:38.994895 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394645437:1394652257, ack 1233000592, win 342, options [nop,nop,TS val 78910775 ecr 78910775], length 6820
11:52:39.035093 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394652257, win 1878, options [nop,nop,TS val 78910815 ecr 78910775], length 0
11:52:39.035141 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394652257:1394662237, ack 1233000592, win 342, options [nop,nop,TS val 78910815 ecr 78910815], length 9980
11:52:39.074820 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394662237, win 1836, options [nop,nop,TS val 78910855 ecr 78910815], length 0
11:52:39.074842 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394662237:1394677717, ack 1233000592, win 342, options [nop,nop,TS val 78910855 ecr 78910855], length 15480
11:52:39.114860 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394677717, win 1771, options [nop,nop,TS val 78910895 ecr 78910855], length 0
11:52:39.114880 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394677717:1394694517, ack 1233000592, win 342, options [nop,nop,TS val 78910895 ecr 78910895], length 16800
11:52:39.154880 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394694517, win 1702, options [nop,nop,TS val 78910935 ecr 78910895], length 0
11:52:39.154897 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394694517:1394711317, ack 1233000592, win 342, options [nop,nop,TS val 78910935 ecr 78910935], length 16800
11:52:39.194830 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394711317, win 1633, options [nop,nop,TS val 78910975 ecr 78910935], length 0
11:52:39.194842 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394711317:1394728117, ack 1233000592, win 342, options [nop,nop,TS val 78910975 ecr 78910975], length 16800
11:52:39.234790 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394728117, win 1564, options [nop,nop,TS val 78911015 ecr 78910975], length 0
11:52:39.234826 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394728117:1394740847, ack 1233000592, win 342, options [nop,nop,TS val 78911015 ecr 78911015], length 12730
11:52:39.274844 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394740847, win 1511, options [nop,nop,TS val 78911055 ecr 78911015], length 0
11:52:39.274862 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394740847:1394744797, ack 1233000592, win 342, options [nop,nop,TS val 78911055 ecr 78911055], length 3950
11:52:39.314819 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394744797, win 1493, options [nop,nop,TS val 78911095 ecr 78911055], length 0
11:52:39.314841 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394744797:1394761597, ack 1233000592, win 342, options [nop,nop,TS val 78911095 ecr 78911095], length 16800
11:52:39.354806 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394761597, win 1424, options [nop,nop,TS val 78911135 ecr 78911095], length 0
11:52:39.354823 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394761597:1394778397, ack 1233000592, win 342, options [nop,nop,TS val 78911135 ecr 78911135], length 16800
11:52:39.394793 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394778397, win 1355, options [nop,nop,TS val 78911175 ecr 78911135], length 0
11:52:39.394814 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394778397:1394795197, ack 1233000592, win 342, options [nop,nop,TS val 78911175 ecr 78911175], length 16800
11:52:39.434759 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394795197, win 1286, options [nop,nop,TS val 78911215 ecr 78911175], length 0
11:52:39.434800 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394795197:1394811997, ack 1233000592, win 342, options [nop,nop,TS val 78911215 ecr 78911215], length 16800
11:52:39.474748 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394811997, win 1217, options [nop,nop,TS val 78911255 ecr 78911215], length 0
11:52:39.474760 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394811997:1394824597, ack 1233000592, win 342, options [nop,nop,TS val 78911255 ecr 78911255], length 12600
11:52:39.514771 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394824597, win 1164, options [nop,nop,TS val 78911295 ecr 78911255], length 0
11:52:39.514789 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394824597:1394828797, ack 1233000592, win 342, options [nop,nop,TS val 78911295 ecr 78911295], length 4200
11:52:39.554833 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394828797, win 1145, options [nop,nop,TS val 78911335 ecr 78911295], length 0
11:52:39.554853 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394828797:1394845597, ack 1233000592, win 342, options [nop,nop,TS val 78911335 ecr 78911335], length 16800
11:52:39.594805 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394845597, win 1076, options [nop,nop,TS val 78911375 ecr 78911335], length 0
11:52:39.594830 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394845597:1394862397, ack 1233000592, win 342, options [nop,nop,TS val 78911375 ecr 78911375], length 16800
11:52:39.634725 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394862397, win 1007, options [nop,nop,TS val 78911415 ecr 78911375], length 0
11:52:39.634744 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394862397:1394879197, ack 1233000592, win 342, options [nop,nop,TS val 78911415 ecr 78911415], length 16800
11:52:39.674776 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394879197, win 938, options [nop,nop,TS val 78911455 ecr 78911415], length 0
11:52:39.674789 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394879197:1394895997, ack 1233000592, win 342, options [nop,nop,TS val 78911455 ecr 78911455], length 16800
11:52:39.714767 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394895997, win 869, options [nop,nop,TS val 78911495 ecr 78911455], length 0
11:52:39.714876 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394895997:1394910697, ack 1233000592, win 342, options [nop,nop,TS val 78911495 ecr 78911495], length 14700
11:52:39.754771 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394910697, win 808, options [nop,nop,TS val 78911535 ecr 78911495], length 0
11:52:39.754788 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394910697:1394912797, ack 1233000592, win 342, options [nop,nop,TS val 78911535 ecr 78911535], length 2100
11:52:39.794797 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394912797, win 797, options [nop,nop,TS val 78911575 ecr 78911535], length 0
11:52:39.794817 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394912797:1394929597, ack 1233000592, win 342, options [nop,nop,TS val 78911575 ecr 78911575], length 16800
11:52:39.834832 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394929597, win 728, options [nop,nop,TS val 78911615 ecr 78911575], length 0
11:52:39.834853 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394929597:1394947417, ack 1233000592, win 342, options [nop,nop,TS val 78911615 ecr 78911615], length 17820
11:52:39.874741 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394947417, win 655, options [nop,nop,TS val 78911655 ecr 78911615], length 0
11:52:39.874759 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394947417:1394963437, ack 1233000592, win 342, options [nop,nop,TS val 78911655 ecr 78911655], length 16020
11:52:39.914743 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394963437, win 589, options [nop,nop,TS val 78911695 ecr 78911655], length 0
11:52:39.914756 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394963437:1394980117, ack 1233000592, win 342, options [nop,nop,TS val 78911695 ecr 78911695], length 16680
11:52:39.954818 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394980117, win 520, options [nop,nop,TS val 78911735 ecr 78911695], length 0
11:52:39.954830 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394980117:1394996917, ack 1233000592, win 342, options [nop,nop,TS val 78911735 ecr 78911735], length 16800
11:52:39.994802 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394996917, win 451, options [nop,nop,TS val 78911775 ecr 78911735], length 0
11:52:39.995152 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394996917:1394996927, ack 1233000592, win 342, options [nop,nop,TS val 78911775 ecr 78911775], length 10
11:52:40.035846 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1394996927, win 451, options [nop,nop,TS val 78911816 ecr 78911775], length 0
11:52:40.035915 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1394996927:1395013717, ack 1233000592, win 342, options [nop,nop,TS val 78911816 ecr 78911816], length 16790
11:52:40.075794 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1395013717, win 374, options [nop,nop,TS val 78911856 ecr 78911816], length 0
11:52:40.075829 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1395013717:1395030517, ack 1233000592, win 342, options [nop,nop,TS val 78911856 ecr 78911856], length 16800
11:52:40.115847 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1395030517, win 305, options [nop,nop,TS val 78911896 ecr 78911856], length 0
11:52:40.115866 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1395030517:1395047317, ack 1233000592, win 342, options [nop,nop,TS val 78911896 ecr 78911896], length 16800
11:52:40.155703 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1395047317, win 174, options [nop,nop,TS val 78911936 ecr 78911896], length 0
11:52:40.155752 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1395047317:1395064117, ack 1233000592, win 342, options [nop,nop,TS val 78911936 ecr 78911936], length 16800
11:52:40.195132 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1395064117, win 43, options [nop,nop,TS val 78911976 ecr 78911936], length 0
11:52:40.435748 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [P.], seq 1395064117:1395069621, ack 1233000592, win 342, options [nop,nop,TS val 78912216 ecr 78911976], length 5504
11:52:40.435782 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1395069621, win 0, options [nop,nop,TS val 78912216 ecr 78912216], length 0
11:52:40.670661 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [.], ack 1233000592, win 342, options [nop,nop,TS val 78912451 ecr 78912216], length 0
11:52:40.670674 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1395069621, win 0, options [nop,nop,TS val 78912451 ecr 78912216], length 0
11:52:41.141703 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [.], ack 1233000592, win 342, options [nop,nop,TS val 78912922 ecr 78912451], length 0
11:52:42.083643 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [.], ack 1233000592, win 342, options [nop,nop,TS val 78913864 ecr 78912451], length 0
11:52:42.083655 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1395069621, win 0, options [nop,nop,TS val 78913864 ecr 78912216], length 0
11:52:43.967506 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [.], ack 1233000592, win 342, options [nop,nop,TS val 78915748 ecr 78913864], length 0
11:52:43.967532 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1395069621, win 0, options [nop,nop,TS val 78915748 ecr 78912216], length 0
11:52:47.739259 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [.], ack 1233000592, win 342, options [nop,nop,TS val 78919520 ecr 78915748], length 0
11:52:47.739274 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1395069621, win 0, options [nop,nop,TS val 78919520 ecr 78912216], length 0
11:52:55.275863 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [.], ack 1233000592, win 342, options [nop,nop,TS val 78927056 ecr 78919520], length 0
11:52:55.275931 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [.], ack 1395069621, win 0, options [nop,nop,TS val 78927056 ecr 78912216], length 0
抓取到的前三個數據包是 blocking_client 與 blocking_server 建立三次握手的過程。
11:52:35.907381 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [S], seq 1394135076, win 43690, options [mss 65495,sackOK,TS val 78907688 ecr 0,nop,wscale 7], length 0
20:32:21.261484 IP 127.0.0.1.3000 > 127.0.0.1.40846: Flags [S.], seq 1233000591, ack 1394135077, win 43690, options [mss 65495,sackOK,TS val 78907688 ecr 78907688,nop,wscale 7], length 0
11:52:35.907441 IP 127.0.0.1.40846 > 127.0.0.1.3000: Flags [.], ack 1233000592, win 342, options [nop,nop,TS val 78907688 ecr 78907688], length 0
示意圖如下：
當每次 blocking_client 給 blocking_server 發數據以後，blocking_server 會應答 blocking_server，在每次應答的數據包中會帶上自己的當前可用 TCP 窗口大小（看上文中結果從 127.0.0.1.3000 > 127.0.0.1.40846 方向的數據包的 win 欄位大小變化），由於 TCP 流量控制和擁賽控制機制的存在，blocking_server 端的 TCP 窗口大小短期內會慢慢增加，後面隨著接收緩衝區中數據積壓越來越多， TCP 窗口會慢慢變小，最終變為 0。
另外，細心的讀者如果實際去做一下這個實驗會發現一個現象，即當 tcpdump 已經顯示對端的 TCP 窗口是 0 時， blocking_client 仍然可以繼續發送一段時間的數據，此時的數據已經不是在發往對端，而是逐漸填滿到本端的內核發送緩衝區中去了，這也驗證了 send 函數實際上是往內核緩衝區中拷貝數據這一行為。
socket 非阻塞模式下的 send 行為
我們再來驗證一下非阻塞 socket 的 send 行為，server 端的代碼不變，我們將 blocking_client.cpp 中 socket 設置成非阻塞的，修改後的代碼如下：
/**
* 驗證非阻塞模式下send函數的行為，client端，nonblocking_client.cpp
* zhangyl 2018.12.17
*/
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <iostream>
#include <string.h>
#include <stdio.h>
#include <fcntl.h>
#include <errno.h>
#define SERVER_ADDRESS "127.0.0.1"
#define SERVER_PORT 3000
#define SEND_DATA "helloworld"
int main(int argc, char* argv[])
{
//1.創建一個socket
int clientfd = socket(AF_INET, SOCK_STREAM, 0);
if (clientfd == -1)
{
std::cout << "create client socket error." << std::endl;
return -1;
}
//2.連接伺服器
struct sockaddr_in serveraddr;
serveraddr.sin_family = AF_INET;
serveraddr.sin_addr.s_addr = inet_addr(SERVER_ADDRESS);
serveraddr.sin_port = htons(SERVER_PORT);
if (connect(clientfd, (struct sockaddr *)&serveraddr, sizeof(serveraddr)) == -1)
{
std::cout << "connect socket error." << std::endl;
close(clientfd);
return -1;
}
//連接成功以後，我們再將 clientfd 設置成非阻塞模式，
//不能在創建時就設置，這樣會影響到 connect 函數的行為
int oldSocketFlag = fcntl(clientfd, F_GETFL, 0);
int newSocketFlag = oldSocketFlag | O_NONBLOCK;
if (fcntl(clientfd, F_SETFL, newSocketFlag) == -1)
{
close(clientfd);
std::cout << "set socket to nonblock error." << std::endl;
return -1;
}
//3. 不斷向伺服器發送數據，或者出錯退出
int count = 0;
while (true)
{
int ret = send(clientfd, SEND_DATA, strlen(SEND_DATA), 0);
if (ret == -1)
{
//非阻塞模式下send函數由於TCP窗口太小發不出去數據，錯誤碼是EWOULDBLOCK
if (errno == EWOULDBLOCK)
{
std::cout << "send data error as TCP Window size is too small." << std::endl;
continue;
}
else if (errno == EINTR)
{
//如果被信號中斷，我們繼續重試
std::cout << "sending data interrupted by signal." << std::endl;
continue;
}
else
{
std::cout << "send data error." << std::endl;
break;
}
}
if (ret == 0)
{
//對端關閉了連接，我們也關閉
std::cout << "send data error." << std::endl;
close(clientfd);
break;
}
else
{
count ++;
std::cout << "send data successfully, count = " << count << std::endl;
}
}
//5. 關閉socket
close(clientfd);
return 0;
}
編譯 nonblocking_client.cpp 得到可執行程序 nonblocking_client：
g++ -g -o nonblocking_client nonblocking_client.cpp
運行 nonblocking_client，運行一段時間後，由於對端和本端的 TCP 窗口已滿，數據發不出去了，但是 send 函數不會阻塞，而是立即返回，返回值是 -1（Windows 系統上 返回 SOCKET_ERROR，這個宏的值也是 -1），此時得到錯誤碼是 EWOULDBLOCK。執行結果如下：
socket 阻塞模式下的 recv 行為
在了解了 send 函數的行為，我們再來看一下阻塞模式下的 recv 函數行為。伺服器端代碼不需要修改，我們修改一下客戶端代碼，如果伺服器端不給客戶端發數據，此時客戶端調用 recv 函數執行流會阻塞在 recv 函數調用處。繼續修改一下客戶端代碼：
/**
* 驗證阻塞模式下recv函數的行為，client端，blocking_client_recv.cpp
* zhangyl 2018.12.17
*/
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <iostream>
#include <string.h>
#define SERVER_ADDRESS "127.0.0.1"
#define SERVER_PORT 3000
#define SEND_DATA "helloworld"
int main(int argc, char* argv[])
{
//1.創建一個socket
int clientfd = socket(AF_INET, SOCK_STREAM, 0);
if (clientfd == -1)
{
std::cout << "create client socket error." << std::endl;
return -1;
}
//2.連接伺服器
struct sockaddr_in serveraddr;
serveraddr.sin_family = AF_INET;
serveraddr.sin_addr.s_addr = inet_addr(SERVER_ADDRESS);
serveraddr.sin_port = htons(SERVER_PORT);
if (connect(clientfd, (struct sockaddr *)&serveraddr, sizeof(serveraddr)) == -1)
{
std::cout << "connect socket error." << std::endl;
close(clientfd);
return -1;
}
//直接調用recv函數，程序會阻塞在recv函數調用處
char recvbuf[32] = {0};
int ret = recv(clientfd, recvbuf, 32, 0);
if (ret > 0)
{
std::cout << "recv successfully." << std::endl;
}
else
{
std::cout << "recv data error." << std::endl;
}
//5. 關閉socket
close(clientfd);
return 0;
}
編譯 blocking_client_recv.cpp 並使用啟動，我們發現程序既沒有列印 recv 調用成功的信息也沒有調用失敗的信息，將程序中斷下來，使用 bt 命令查看此時的調用堆棧，發現程序確實阻塞在 recv 函數調用處。
[root@localhost testsocket]# g++ -g -o blocking_client_recv blocking_client_recv.cpp
[root@localhost testsocket]# gdb blocking_client_recv
Reading symbols from /root/testsocket/blocking_client_recv...done.
(gdb) r
Starting program: /root/testsocket/blocking_client_recv
^C
Program received signal SIGINT, Interrupt.
0x00007ffff72f119d in recv () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-196.el7_4.2.x86_64 libgcc-4.8.5-16.el7_4.2.x86_64 libstdc++-4.8.5-16.el7_4.2.x86_64
(gdb) bt
#0 0x00007ffff72f119d in recv () from /lib64/libc.so.6
#1 0x0000000000400b18 in main (argc=1, argv=0x7fffffffe588) at blocking_client_recv.cpp:40
socket 非阻塞模式下的 recv 行為
非阻塞模式下如果當前無數據可讀，recv 函數將立即返回，返回值為 -1，錯誤碼為 EWOULDBLOCK。將客戶端代碼修成一下：
/**
* 驗證阻塞模式下recv函數的行為，client端，blocking_client_recv.cpp
* zhangyl 2018.12.17
*/
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <iostream>
#include <string.h>
#include <stdio.h>
#include <fcntl.h>
#include <errno.h>
#define SERVER_ADDRESS "127.0.0.1"
#define SERVER_PORT 3000
#define SEND_DATA "helloworld"
int main(int argc, char* argv[])
{
//1.創建一個socket
int clientfd = socket(AF_INET, SOCK_STREAM, 0);
if (clientfd == -1)
{
std::cout << "create client socket error." << std::endl;
return -1;
}
//2.連接伺服器
struct sockaddr_in serveraddr;
serveraddr.sin_family = AF_INET;
serveraddr.sin_addr.s_addr = inet_addr(SERVER_ADDRESS);
serveraddr.sin_port = htons(SERVER_PORT);
if (connect(clientfd, (struct sockaddr *)&serveraddr, sizeof(serveraddr)) == -1)
{
std::cout << "connect socket error." << std::endl;
close(clientfd);
return -1;
}
//連接成功以後，我們再將 clientfd 設置成非阻塞模式，
//不能在創建時就設置，這樣會影響到 connect 函數的行為
int oldSocketFlag = fcntl(clientfd, F_GETFL, 0);
int newSocketFlag = oldSocketFlag | O_NONBLOCK;
if (fcntl(clientfd, F_SETFL, newSocketFlag) == -1)
{
close(clientfd);
std::cout << "set socket to nonblock error." << std::endl;
return -1;
}
//直接調用recv函數，程序會阻塞在recv函數調用處
while (true)
{
char recvbuf[32] = {0};
int ret = recv(clientfd, recvbuf, 32, 0);
if (ret > 0)
{
//收到了數據
std::cout << "recv successfully." << std::endl;
}
else if (ret == 0)
{
//對端關閉了連接
std::cout << "peer close the socket." << std::endl;
break;
}
else if (ret == -1)
{
if (errno == EWOULDBLOCK)
{
std::cout << "There is no data available now." << std::endl;
}
else if (errno == EINTR)
{
//如果被信號中斷了，則繼續重試recv函數
std::cout << "recv data interrupted by signal." << std::endl;
} else
{
//真的出錯了
break;
}
}
}
//5. 關閉socket
close(clientfd);
return 0;
}
執行結果與我們預期的一模一樣， recv 函數在無數據可讀的情況下並不會阻塞情緒，所以程序會一直有「There is no data available now.」相關的輸出。
本文首發於『easyserverdev』公眾號，歡迎關注，轉載請保留版權信息。
歡迎加入高性能伺服器開發 QQ 群一起交流： 578019391 。
推薦閱讀：
