目錄
4.2.4 client_ext_hello和client_ext_hello_answer
4.3.5 "開始上傳文件"請求及其響應（OP_STARTUPLOADREQ）
第四章 P2P消息
4.1 libed2k p2p 連接工作流
處理P2P消息的邏輯被封裝在 `base_connection`和`peer_connection`對象中，調用棧如下:
session::listen_on(int port, const char* net_interface /*= 0*/) <--在這裏開始監聽TCP連接
|---void session_impl::open_listen_port()
|---async_accept
|---session_impl::on_accept_connection <--TCP 建立
|---session_impl::incoming_connection(boost::shared_ptr<tcp::socket> const& s)
|---new peer_connection(*this, s, endp, NULL)
|---setup P2P message hander
|---peer_connection::do_read
|---base_connection::do_read(); //握手，設置選項等。
| |---base_connection::on_read_header
| |---base_connection::on_read_packet
| |---find handler by type and protocol in header
| |---call handler() <-- 處理收到的數據
或者
|---peer_connection::receive_data() //當文件傳輸正在進行時
|---peer_connection::on_receive_data(error, bytes_transferred)
|--process data received. <-- 處理收到的數據
點對點消息處理函數的設置在`peer_connection::reset()`函數中，接下來分各節說明P2P消息。
4.2 HELLO和HELLO answer
4.2.1消息定義
(From EMule Protocol 第六章 6.4.1)
This message is the first message in the handshake between two e-mule clients. This message is very much like the server login message (see in section 6.2.1). Both messages have the same type code (0x01), they are the only messages in the protocol that have overlapping type code. Both messages provide the same data and even in the same order. There are two main differences: the client hello message begins with a user hash size field while the server login message immediately begins with the user hash value, also the client hello message ends with additional server IP and port information which is not relevant for the server login message.
|
Name |
Size in bytes |
Default Value |
Comment |
|
Protocol |
1 |
0xE3 |
|
|
Size |
4 |
The size of the message in bytes not including the header and size fields |
|
|
Type |
1 |
0x01 |
The value of the OP HELLO opcode |
|
User Hash size |
1 |
16 |
The size of the user hash field |
|
以下部分和Hello answer消息的結構一致 |
|||
|
User Hash |
16 |
TBD |
|
|
Client ID |
4 |
0 |
TBD |
|
TCP Port |
2 |
4662 |
The TCP port used by the client, configurable |
|
Tag Count |
4 |
4 |
The number of tags following in the message |
|
Tag list |
varies |
NA |
A list of tags specifying remote client’s properties |
|
Server IP |
4 |
NA |
The IP of the server to which the client is connected |
|
Server TCP Port |
2 |
NA |
The TCP port on which the server listens |
There are three types of tags that may appear in the tag-list. The port tag is optional and usually not provided. Tag encoding rules are described is detail an the beginning of this chapter.
|
Name |
Tag name |
Tag Type |
Comment |
|
Username |
Integer, 0x01 |
String |
|
|
Version |
Integer 0x11 |
String |
|
|
Port |
Integer 0x0F |
Integer |
HELLO Answer (From EMule Protocol 6.4.1)
Sent as an answer to a Hello message. Contains exactly the same fields as the hello message except for the message type.
|
Name |
Size in bytes |
Default Value |
Comment |
|
Protocol |
1 |
0xE3 |
|
|
Size |
4 |
The size of the message in bytes not including the header and size fields |
|
|
Type |
1 |
0x4C |
The value of the OP HELLOANSWER op-code |
|
Hello fields |
The same fields as in the hello message starting with the user hash |
4.2.2 收取Hello和helloAnswer消息
message handlers在"peer_connection::reset()"中設置：
add_handler(std::make_pair(OP_HELLO, OP_EDONKEYPROT),
boost::bind(&peer_connection::on_hello, this, _1));
add_handler(get_proto_pair<client_hello_answer>(),
boost::bind(&peer_connection::on_hello_answer, this, _1));
從網絡流中解析HELLO消息peer_connection::on_hello:
void peer_connection::on_hello(const error_code& error)
{
if (!error){
DECODE_PACKET(client_hello, hello);
// store user info
m_hClient = hello.m_hClient;
m_options.m_nPort = hello.m_network_point.m_nPort;
//解析hello消息中的tag_list並存放到m_options對應的項中
parse_misc_info(hello.m_list);
DBG("hello {" << " server point = " << hello.m_server_network_point << " network point = " << hello.m_network_point << "} <== " << m_remote);
//在CALLBACK列表中尋找當前連接的用戶，如果找到就從列表中移除並返回文件md4hash
//前面有說明CALLBACK已經不被大部分的服務器所支持，這裏我們跳過對它的說明。
md4_hash file_hash = m_ses.callbacked_lowid(hello.m_network_point.m_nIP);
if (file_hash != md4_hash::invalid)
{
DBG("lowid peer detected for " << file_hash.toString());
m_active = true;
attach_to_transfer(file_hash);
}
//收到了對方發送的hello消息，給他一個ACK。
write_hello_answer();
}
else
{
ERR("hello packet received error " << error.message());
}
}
反序列化流到`client_hello`對象：
DECODE_PACKET with T=client_hello, t = hello in following code:
template<typename T>
bool decode_packet(T& t)
{
try{
if (!m_in_container.empty()){
boost::iostreams::stream_buffer<base_connection::Device> buffer(
&m_in_container[0],
m_in_header.m_size - 1);
std::istream in_array_stream(&buffer);
archive::ed2k_iarchive ia(in_array_stream);
ia >> t;
}
}catch(libed2k_exception& e){
DBG("Error on conversion " << e.what());
return (false);
}
return (true);
}
"client_hello"序列化:
struct client_hello : public client_hello_answer
{
boost::uint8_t m_nHashLength; //!< clients hash length
//...
template<typename Archive>
void serialize(Archive& ar)
{
ar & m_nHashLength;
client_hello_answer::serialize(ar);
}
};
`client_hello_answer`類型定義:
struct client_hello_answer
{
md4_hash m_hClient;
net_identifier m_network_point;
tag_list<boost::uint32_t> m_list;
net_identifier m_server_network_point;
//...
template<typename Archive>
void serialize(Archive& ar)
{
ar & m_hClient & m_network_point & m_list & m_server_network_point;
}
};
client_hello和client_hello_answer唯一的不同是增加了一個字節的m_nHashLength。
下面是從網絡流中讀取client_hello_answer：
void peer_connection::on_hello_answer(const error_code& error)
{
if (!error){
DECODE_PACKET(client_hello_answer, packet);
//解析對方hello響應消息中的tag_list並存放到m_options對應的項中
parse_misc_info(packet.m_list);
m_hClient = packet.m_hClient;
DBG("hello answer {name: " << m_options.m_strName << " : mod name: " << m_options.m_strModVersion << ", port: " << m_options.m_nPort << "} <== " << m_remote);
m_ses.m_alerts.post_alert_should(
peer_connected_alert(get_network_point(),
get_connection_hash(),
m_active)
);
//我們主動發起的連接，收到hello answer消息就完成了握手。
//在下面的函數中發起一條“文件請求”消息，傳入我們所需文件的md4 hash。
finalize_handshake();
}
else
{
ERR("hello error " << error.message() << " <== " << m_remote);
}
}
4.2.3 發送hello和helloAnswer消息
在發起連接完成後（peer_connection::on_connect），連接發起方首條發送的消息就是hello:
void peer_connection::write_hello() {
DBG("hello ==> " << m_remote);
const session_settings& settings = m_ses.settings();
client_hello hello(m_ses.settings().user_agent,
net_identifier(m_ses.m_server_connection->client_id(),m_ses.settings().listen_port),
net_identifier(address2int(m_ses.server().address()), m_ses.server().port()),
m_ses.settings().client_name,
m_ses.settings().mod_name,
m_ses.settings().m_version);
// fill special fields
hello.m_nHashLength = MD4_DIGEST_LENGTH;
hello.m_network_point.m_nIP = m_ses.m_server_connection->client_id();
hello.m_network_point.m_nPort = settings.listen_port;
// 這個函數填充hello/helloAnswer消息的tag_list,
// 上面提到過這兩個消息的結構僅有一個HashLength不一樣。
append_misc_info(hello.m_list);
write_struct(hello);
}
接收方收到HELLO消息時發回響應如下：
void peer_connection::write_hello_answer()
{
// prepare hello answer
client_hello_answer cha(m_ses.settings().user_agent,
net_identifier(m_ses.m_server_connection->client_id(),
m_ses.settings().listen_port),
net_identifier(address2int(m_ses.server().address()), m_ses.server().port()),
m_ses.settings().client_name,
m_ses.settings().mod_name,
m_ses.settings().m_version);
//這個函數創建hello/helloAnswer的tag_list
append_misc_info(cha.m_list);
cha.dump();
DBG("hello answer ==> " << m_remote);
write_struct(cha);
//注意下面這個函數在對方發起連接時接收方的on_hello_answer中也有調用。
//在這裏我們是發送方，所以這個函數此時僅創建和發送緩衝區有關的數據結構實例。
finalize_handshake();
}
append_misc_info函數的實現：
void peer_connection::append_misc_info(tag_list<boost::uint32_t>& t) {
//以下選項（mo/mo2）似乎只是爲了兼容性而設計，
//libed2k客戶端並未根據這些選項定製不同策略。
misc_options mo(0);
mo.m_nUnicodeSupport = 1;
mo.m_nDataCompVer = 0;
// support data compression
mo.m_nNoViewSharedFiles = !m_ses.settings().m_show_shared_files;
mo.m_nSourceExchange1Ver = SOURCE_EXCHG_LEVEL;
misc_options2 mo2(0);
mo2.set_captcha();
//允許驗證圖片
mo2.set_large_files();
//允許大文件
mo2.set_source_ext2();
//允許Source Extend
//USER_NAME
t.add_tag(make_string_tag(m_ses.settings().client_name, CT_NAME, true));
//VERSION
t.add_tag(make_typed_tag(m_ses.settings().m_version, CT_VERSION, true));
//以下內容不在emule protocol中，應該是新版服務器所支持的。
//ed2k版本
t.add_tag(make_typed_tag(make_full_ed2k_version(SO_AMULE, m_ses.settings().mod_major,
m_ses.settings().mod_minor,
m_ses.settings().mod_build),
CT_EMULE_VERSION,
true));
//options1：UnicodeSupport/data compression/No View Shared Files/SourceExchange
t.add_tag(make_typed_tag(mo.generate(), CT_EMULE_MISCOPTIONS1, true));
//options2
t.add_tag(make_typed_tag(mo2.generate(), CT_EMULE_MISCOPTIONS2, true));
}
4.2.4 client_ext_hello和client_ext_hello_answer
libed2k支持使用OP_EMULEPROT（0xC5）協議，在使用這個協議時支持消息ID爲OP_EMULEINFO(注意和HELLO一樣值爲1)的消息，對應的響應消息是OP_EMULEINFOANSWER(2)。在libed2k中消息體定義如下：
struct client_ext_hello
{
boost::uint16_t m_nVersion;
tag_list<boost::uint32_t> m_list;
template<typename Archive>
void serialize(Archive& ar){ ar & m_nVersion & m_list; }
};
struct client_ext_hello_answer
{
boost::uint16_t m_nVersion;
tag_list<boost::uint32_t> m_list;
template<typename Archive>
void serialize(Archive& ar){ ar & m_nVersion & m_list; }
};
4.3 文件請求
在上一節中中提到，當收到對方的helloAnswer消息（即完成握手）時將發送一條文件請求消息。這一節將說明和文件請求有關的協議內容。
4.3.1 文件請求概述
(from emule protocol 4.3)
As already mentioned a separate connection is created for each [client,file] pair. Immediately after the connection establishment the client sends several query messages regarding the file it wishes to download. A typical, successful scenario is demonstrated in 4.3.
Figure 4.3: File request
Basic message exchange
The basic message exchange is composed from four messages: A sends a file request message (section 6.4.18) immediately followed by a requested file ID message (section 6.4.17). B replies to the file request by a file request answer (section 6.4.15) and to the requested file ID message by a file status (section 6.4.18) message. I couldn’t find any reason to divide the information sent in these messages to four messages, it could easily be handled by two messages (a request and a reply).
The extended protocol adds two messages to this sequence a sources request (section 6.5.6) and a sources answer section 6.5.7. This extension is used to pass B’s sources (in case B is currently downloading the file) to A. To elaborate, there is no requirement that B completes to download a file before it can send file parts to other clients, B can send to A any part it has completed to download even when it has only a small fragment of the file.
File not found scenario
When A requests a file from B but B doesn’t have this file in its shared file list. B skips the file request answer message and send a file not found message (section 6.4.16), immediately after the requested file ID message as demonstrated in figure 4.4.
Figure 4.4: File request failure - file not found
Enlisting to the upload queue
In the case where B has the requested file but his upload queue is not empty which means that there are clients that are downloading files and there are probably also clients in the the upload queue A and B perform the full handshake described in figure 4.3 but when A request B to start uploading the file, B adds A to his upload queue and replies with a queue ranking message (section 6.5.4) which contains A’s position in B’s upload queue. Figure 4.5 illustrates this sequence.
Figure 4.5: File request waiting queue
Upload queue management
For each uploaded file the client maintains an upload priority queue. The priority of each client in the queue is calculated on the basis of the client’s time in the queue and a priority modifier. At the head of the queue are clients which have the highest score. The score is calculated using the following formula: score = (rating ∗ seconds in the queue)/100 or ∞ in case the downloading client is defined as a friend. The initial rating value is 100 except for banned users which receive 0 rating (and thus are revented from reaching the top of the queue). The rating is modified either by the downloading client’s credit (which ranges from 1 - 10) or by the uploaded file priority (0.2 - 1.8) which is set by the uploading client. When a client’s score is higher than the core of the rest of the clients it starts downloading the file.
A client will continue to download a file until one of the following conditions occurs:
- The uploading client was terminated by the user
- The downloading client has got all the parts it needs for the file
- The downloading client was preempted by another downloading client which has higher priority than his.
In order to allow a client which just started downloading to get a few megabytes of data before it is preempted, eMule boosts the initial rating of a downloading client to 200 for the first 15 minutes of its download.
Reaching the top of the upload queue
When A reaches the top of B’s upload queue, B connects to A, performs the initial handshake and then sends an accept upload request message (section 6.4.11). A can now choose either to continue and download the file by sending a request parts message or to cancel (in case it already got the part from another source) by sending he cancel transfer message (section 6.4.12). Figure 4.6 illustrates these options.
Figure 4.6: File request resume download
4.3.2 “文件請求”消息和響應
（from emule protocol 6.4.14File request）
A message sent from a client requesting a file from another client. The message contains the ID of the requested file and a status field describing which parts are already downloaded. The part status field eMule allows clients to download file parts from other clients even when the providing client has not yet completed downloading the requested file. the Part status field helps distinguishing between a file that simply doesn’t exist on the requested client and a file that is only partially downloaded on by it. In case the file doesn’t exist then the value of Part status is 0. In case the file is partially downloaded, the first 2 bytes are an integer giving the number of parts already downloaded and the last byte is a bitmap which indicates which eighth of the file is completely downloaded (by setting the matching bit to 1).存疑：這裏的說明是文件完全未下載0；文件部分下載時爲xx-x，文件分成八塊，頭兩個字節爲已下載的塊數而最後一個字節的每一位代表表示哪個塊已經下載完成。如果按照這裏的說明，根本無法處理8個分片（大約72MB）以上的文件。另外注意libed2k並未填充這兩個可選項。
|
Name |
Size in bytes |
Default Value |
Comment |
|
Protocol |
1 |
0xE3 |
|
|
Size |
4 |
The size of the message in bytes not including the header and size fields |
|
|
Type |
1 |
0x58 |
The value of the OP FILEREQUEST opcode |
|
File ID |
16 |
NA |
Unique file ID |
|
Part Status |
3 |
NA |
（注：libed2k並未使用這個可選項）Optional, sent if the extended request version indicated in the eMule info message is greater than zero. The file significance is explained in this section |
|
Source count |
2 |
NA |
（注：libed2k並未使用這個可選項）Optional, sent if the extended request version indicated in the eMule info message is greater than one.Indicated the current number of sources for this file |
文件請求消息的迴應消息之一File request answer：
A file request answer, sent as a reply to a file request message. This message is only one of
the possible replies to a file request more details in section 4.3
|
Name |
Size in bytes |
Default Value |
Comment |
|
Protocol |
1 |
0xE3 |
|
|
Size |
4 |
The size of the message in bytes not including the header and size fields |
|
|
Type |
1 |
0x59 |
The value of the OP FILEREQANSWER op-code |
|
File ID |
16 |
NA |
A unique file ID (hash) |
|
Name length |
2 |
NA |
The filename length |
|
Filename |
Varies |
NA |
The filename (in the length specified) |
文件請求消息的迴應消息之二File not found:
（from emule protocol 6.4.16 File not found）
This message replies a file request and indicates that a requested file or part of a file was not
found on the client.
|
Name |
Size in bytes |
Default Value |
Comment |
|
Protocol |
1 |
0xE3 |
|
|
Size |
4 |
The size of the message in bytes not including the header and size fields |
|
|
Type |
1 |
0x48 |
The value of the OP_FILEREQANSNOFIL opcode |
|
File ID |
16 |
NA |
The none existent file ID |
|
Part Status |
3 |
NA |
（注：libed2k未使用這個項）Explained in file request (section 6.4.14) |
下載方（請求文件者）連接上對方的時候發送“文件請求”：
void peer_connection::write_file_request(const md4_hash& file_hash)
{
DBG("file request " << file_hash << " ==> " << m_remote);
client_file_request fr;
fr.m_hFile = file_hash;
write_struct(fr);
}
在libed2k的peer_connection::reset中設置client_file_request的消息處理函數爲：
add_handler(get_proto_pair<client_file_request>(),
boost::bind(&peer_connection::on_file_request, this, _1));
上傳方（文件提供者）收到“文件請求”消息時的處理：
void peer_connection::on_file_request(const error_code& error){
if (!error){
DECODE_PACKET(client_file_request, fr);
DBG("file request " << fr.m_hFile << " <== " << m_remote);
if (attach_to_transfer(fr.m_hFile)){
//在transfer列表中找到了對方（下載方）需要的文件
boost::shared_ptr<transfer> t = m_transfer.lock();
write_file_answer(t->hash(), t->name());
}else{
//沒有對方要的文件
write_no_file(fr.m_hFile);
}
}else{
ERR("file request error " << error.message() << " <== " << m_remote);
}
}
當上傳方（文件提供者）收到對方發送文件請求且對方請求的文件存在時，發送響應"OP_REQFILENAMEANSWER"將持有的文件名告知對方：
void peer_connection::write_file_answer( const md4_hash& file_hash,
const std::string& filename)
{
DBG("file answer " << file_hash << ", " << filename << " ==> " << m_remote);
client_file_answer fa;
fa.m_hFile = file_hash;
fa.m_filename.m_collection = filename;
write_struct(fa);
}
當上傳方（文件提供者）收到對方發送文件請求且對方請求的文件不存在時，發送OP_FILEREQANSNOFIL響應：
void peer_connection::write_no_file(const md4_hash& file_hash)
{
DBG("no file " << file_hash << " ==> " << m_remote);
client_no_file nf;
nf.m_hFile = file_hash;
write_struct(nf);
}
當下載方（請求文件者）收到對方發送的client_file_answer時的處理，在reset函數中：
add_handler(get_proto_pair<client_file_answer>(),
boost::bind(&peer_connection::on_file_answer, this, _1));
peer_connection::on_file_answer的實現：
void peer_connection::on_file_answer(const error_code& error)
{
if (!error) {
DECODE_PACKET(client_file_answer, fa);
DBG("file answer " << fa.m_hFile << ", " << fa.m_filename.m_collection << " <== " << m_remote);
//發起“文件狀態”請求
write_filestatus_request(fa.m_hFile);
} else {
ERR("file answer error " << error.message() << " <== " << m_remote);
}
}
注意libed2k實際工作流程和圖4-3的流程不同，在圖4-3中，fileStatus請求和fileRequest是併發的。在libed2k中的實現是順序執行，在發送fileRequest收到迴應後才發送fileStatus請求。
4.3.3 "文件狀態"請求
收到client_file_answer後，客戶端發送一條“文件狀態”請求，它長得像下面這樣：
（emule protocol 6.4.17Requested file ID）
A file download request message. The message only specifies the File ID.
|
Name |
Size in bytes |
Default Value |
Comment |
|
Protocol |
1 |
0xE3 |
|
|
Size |
4 |
The size of the message in bytes not including the header and size fields |
|
|
Type |
1 |
0x4E |
The value of the OP SETREQFILEID opcode |
|
File ID |
16 |
NA |
The ID of the requested file |
在ed2k中文件下載方（請求文件者）發起“文件狀態”請求的代碼如下：
void peer_connection::write_filestatus_request(const md4_hash& file_hash)
{
DBG("request file status " << file_hash << " ==> " << m_remote);
client_filestatus_request fr;
fr.m_hFile = file_hash;
write_struct(fr);
}
文件上傳方（文件提供者）收到“文件狀態”請求的處理，在reset函數中設置處理函數：
add_handler(/*OP_SETREQFILEID*/get_proto_pair<client_filestatus_request>(),
boost::bind(&peer_connection::on_filestatus_request, this, _1));
peer_connection::on_filestatus_request的實現：
void peer_connection::on_filestatus_request(const error_code& error) {
if (!error){
DECODE_PACKET(client_filestatus_request, fr);
DBG("file status request " << fr.m_hFile << " <== " << m_remote);
boost::shared_ptr<transfer> t = m_transfer.lock();
if (!t)
return;
if (t->hash() == fr.m_hFile){
write_file_status(t->hash(), t->have_pieces());
}else{
write_no_file(fr.m_hFile);
disconnect(errors::file_unavaliable, 2);
}
}else{
ERR("file status request error " << error.message() << " <== " << m_remote);
}
}
可以見到，如果transfer對象不存在則直接斷開。然後有文件的時候調用write_file_status發送“文件狀態”響應，文件不存在的時候發送的是write_no_file。
注意如果對方的客戶端也是libed2k客戶端，因爲順序發送的關係在沒有文件的時候不會到達write_no_file這裏。但是如emule protocol（已摘抄到本章4.3.1）中說明的，對方客戶端可能會同時發送“文件請求”和“文件狀態”消息，所以這裏還是要提供一條“文件不存在”的響應。
（當有文件分片時）文件上傳方發送“文件狀態”，用於發送它擁有的文件分片信息：
（enule protocol 6.4.18 File status）
Sent as a reply to a requested file ID message in the case the client has the requested file.
|
Name |
Size in bytes |
Default Value |
Comment |
|
Protocol |
1 |
0xE3 |
|
|
Size |
4 |
The size of the message in bytes not including the header and size fields |
|
|
Type |
1 |
0x50 |
The value of the OP FILESTATUS opcode |
|
File ID |
16 |
NA |
The ID of the file whose status is reported |
|
Part Status |
3【2+((分片數+7)/8)字節】或者【2】 |
NA |
Explained in file request (section 6.4.14),在本文檔的（4.3.2）(文檔已過時,現在每一位代表對應分片是否存在，當擁有整個文件時等於0x00 00) |
libed2k中peer_connection::write_file_status的實現：
void peer_connection::write_file_status( const md4_hash& file_hash,
const bitfield& status)
{
DBG("file status " << file_hash << ", [" << bitfield2string(status) << "] ==> " << m_remote);
client_file_status fs;
fs.m_hFile = file_hash;
fs.m_status = status;
write_struct(fs);
}
其中第二參數傳自transfer::have_pieces:
bitfield transfer::have_pieces() const{
int nPieces = num_pieces();
bitfield res(nPieces, false);
for (int p = 0; p < nPieces; ++p)
if (have_piece(p))
res.set_bit(p);
return res;
}
與4.3.2引述的協議文檔中指出，如果文件完全未下載則爲00-0，否則爲[已下載塊數][掩碼]，一個字節的掩碼用於指出哪八分之一是已經下載完成的。實際上當持有這個文件的一部分時Part Status等於：
- 分片總數(2個字節，最大65535)
- 與分片總數對應的位數，已完成分片下載時設定對應的位。
如果持有整個文件則等於0x00 00，這是因爲上傳方擁有所有分片所以自然不需要發送掩碼了。這與協議中的說明完全不一樣，這個協議應該部分已經過時了(猜測應該時最初的協議不支持大文件)。
可以看到目前爲止接收方是不知道文件的分片細節而只能知道對方是不是擁有文件的一部分或者全部，但是從這條消息就可以與自己手動的分片信息對比以確定是否可以從對方那裏獲取自身不存在的那些分片，從而有針對性的處理，例如當對方沒有所需的分片則可以將這個連接授予較低的優先級等等。
獲取文件的分片細節依賴下一節中的“分片哈希請求”。
下載方收到對方發送的File Status的處理：
add_handler(/*OP_FILESTATUS*/get_proto_pair<client_file_status>(),
boost::bind(&peer_connection::on_file_status, this, _1));
peer_connection::on_file_status的實現：
void peer_connection::on_file_status(const error_code& error) {
if (!error){
DECODE_PACKET(client_file_status, fs);
boost::shared_ptr<transfer> t = m_transfer.lock();
if (!t)
return;
if (fs.m_status.size() == 0)
fs.m_status.resize(t->num_pieces(), 1);
if (t->hash() == fs.m_hFile && t->has_picker()){
m_remote_pieces = fs.m_status;
t->picker().inc_refcount(fs.m_status);
if (t->size() < PIECE_SIZE)
write_start_upload(fs.m_hFile);
else if (fs.m_status.count() > 0)
write_hashset_request(fs.m_hFile);
}else{
write_no_file(fs.m_hFile);
disconnect(errors::file_unavaliable, 2);
}
}else{
ERR("file status answer error " << error.message() << " <== " << m_remote);
}
}
當下載方收到對方手上的文件片段時：
- 如果對方的part status傳遞的是0，那麼說明對方擁有整個文件，此時以碎片數（注1）填充1到代表狀態的bitfield。
- 保存對方的碎片信息到peer_connection::m_remote_pieces。
- 否則保存對方的分片塊信息。然後執行：
- 如果文件大小小於一個分片，直接向對方請求文件數據。
- 否則向對方發送client_hashset_request，（OP_HASHSETREQUEST=0x51）。
注1：transfer對象在構建時指定的add_transfer_params中需要指定文件的大小，所以這裏的碎片數（即上面代碼中的t->num_pieces()）是可以確定的，只需要將文件的總大小除以每個文件片的大小即可獲得，所以即便是對方只傳回0x00 00也可以構造出一個代表文件分片狀態的bitfield。
4.3.4 "分片哈希集"消息及響應
通過上一步“文件狀態”消息交換信息，現在下載方已經知道了上傳方有沒有要求的文件，以及他擁有了哪些分片。現在讓上傳方發送他已有碎片的哈希值這樣我們下載完後可以對文件片進行校驗。
OP_HASHSETREQUEST消息的格式如下（emule protocol 6.4.8 Part hashset request）：Send as a request for the hash of all the requested file and for each and every part in the file. The unique file ID and the file hash are explained in section 1.5.
|
Name |
Size in bytes |
Default Value |
Comment |
|
Protocol |
1 |
0xE3 |
|
|
Size |
4 |
The size of the message in bytes not including the header and size fields |
|
|
Type |
1 |
0x51 |
The value of the OP_HASHSETREQUEST opcode |
|
File ID |
16 |
NA |
The file ID of the requested file |
下載方發送這條消息的實現：
void peer_connection::write_hashset_request(const md4_hash& file_hash) {
DBG("request hashset for " << file_hash << " ==> " << m_remote);
client_hashset_request hr;
hr.m_hFile = file_hash;
write_struct(hr);
}
上傳方收到這條消息時的處理：
void peer_connection::on_hashset_request(const error_code& error) {
if (!error){
DECODE_PACKET(client_hashset_request, hr);
DBG("hashset request " << hr.m_hFile << " <== " << m_remote);
boost::shared_ptr<transfer> t = m_transfer.lock();
if (!t)
return;
if (t->hash() == hr.m_hFile){
write_hashset_answer(t->hash(), t->piece_hashses());
}else{
write_no_file(hr.m_hFile);
disconnect(errors::file_unavaliable, 2);
}
}else{
ERR("hashset request error " << error.message() << " <== " << m_remote);
}
}
上傳方收到這條消息時時，將填充和發送OP_HASHSETANSWER消息。消息的結構如下（emule protocol 6.4.9 Part hashset reply)：
A reply to a file hash set request containing the global hash (for all the file) and a hash for
every part of the file. When the processing of this message is complete the client sends a start
upload request.
|
Name |
Size in bytes |
Default Value |
Comment |
|
Protocol |
1 |
0xE3 |
|
|
Size |
4 |
The size of the message in bytes not including the header and size fields |
|
|
Type |
1 |
0x52 |
The value of the OP HASHSETANSWER op-code |
|
File Hash |
16 |
NA |
The hash extracted from all the file |
|
Part Count |
2 |
NA |
The number of parts in the file |
|
Part Hashes |
Varies |
NA |
A hash for each file part - the size of each hash is 16 bytes |
在libed2k中哈希值存放在transfer::transfer_info.m_piece_hashes,類型是std::vector<md4_hash>。關於分片和文件存儲的內容，將在下一章中介紹。
下載方收到消息OP_HASHSETANSWER時將調用peer_connection::on_hashset_answer處理：
void peer_connection::on_hashset_answer(const error_code& error) {
if (!error) {
DECODE_PACKET(client_hashset_answer, ha);
const std::vector<md4_hash>& hashes = ha.m_vhParts.m_collection;
boost::shared_ptr<transfer> t = m_transfer.lock();
if (!t)
return;
if (t->hash() == ha.m_hFile && t->hash() == md4_hash::fromHashset(hashes)) {
t->piece_hashses(hashes);
write_start_upload(t->hash());
}else{
write_no_file(ha.m_hFile);
disconnect(errors::file_unavaliable, 2);
}
}else{
ERR("hashset answer error " << error.message() << " <== " << m_remote);
}
}
md4_hash::fromHashset(hashes)從一堆分片的hash獲得一整個文件的hash，然後將它同將要下載的文件的總md4 hash進行對比，如果正確則填充transfer對象的piece_hashses否則報錯且關閉連接。注意這裏當連接多個用戶下載一個文件時代表下載任務的transfer對象的piece_hashses成員會被多次覆蓋寫入，這裏可以進行一些改進。fromHashset的實現如下：
/*static*/
md4_hash md4_hash::fromHashset(const std::vector<md4_hash>& hashset) {
md4_hash result;
if (hashset.size() > 1) {
MD4_CTX ctx;
MD4_Init(&ctx);
MD4_Update(&ctx,
reinterpret_cast<const boost::uint8_t*>(&hashset[0]),
hashset.size() * MD4_DIGEST_LENGTH);
MD4_Final(result.getContainer(), &ctx);
}
else if (hashset.size() == 1)
{
result = hashset[0];
}
return result;
}
這裏有一個地方需要注意：下載方只有部分文件而且不知道其他缺失的文件分片的哈希值時，從已持有的文件碎片的hashset是不能獲得完整文件的哈希值的。這意味着ed2k網絡裏的每一個參與者都必須擁有他所持有的文件（即使是隻擁有部分文件片）的整個文件的分片信息。所以在下載暫停時必須要保存這些碎片信息到臨時文件，在後面的章節中我們將詳細分析從臨時文件恢復這些信息的方法。
4.3.5 "開始上傳文件"請求及其響應（OP_STARTUPLOADREQ）
在上一節中，文件下載方已經收到了文件分片的hash信息，校對無誤後將調用peer_connection::write_start_upload請求對方的文件，這個函數將發送“開始上傳文件”請求，它長得像下面這樣（emule protocol 6.4.10 Start upload request）：
A start upload request message. This message starts the file download sequence which is
discussed in section 4.3（對應本文檔的4.3.1）.
|
Name |
Size in bytes |
Default Value |
Comment |
|
Protocol |
1 |
0xE3 |
|
|
Size |
4 |
The size of the message in bytes not including the header and size fields |
|
|
Type |
1 |
0x54 |
The value of the OP STARTUPLOADREQ opcode |
|
File ID |
16 |
NA |
The ID of the requested file |
在libed2k中的實現如下：
void peer_connection::write_start_upload(const md4_hash& file_hash) {
DBG("start upload " << file_hash << " ==> " << m_remote);
client_start_upload su;
su.m_hFile = file_hash;
write_struct(su);
}
上傳方對這條消息的響應消息是OP_ACCEPTUPLOADREQ（emule protocol 6.4.11Accept upload request）:
An ack, indicating the upload request was accepted and the uploading client is now waiting for part requests. The message has no fields except the standard eMule message header. This message is part of the part sending protocol.
|
Name |
Size in bytes |
Default Value |
Comment |
|
Protocol |
1 |
0xE3 |
|
|
Size |
4 |
0 |
The size of the message in bytes not including the header and size fields |
|
Type |
1 |
0x55 |
The value of the OP ACCEPTUPLOADREQ opcode |
如上面說明這條消息只是簡單的發送一條應答，告訴對方已經準備好上傳數據了。在libed2k中的實現如下：
void peer_connection::on_start_upload(const error_code& error) {
if (!error) {
DECODE_PACKET(client_start_upload, su);
DBG("start upload " << su.m_hFile << " <== " << m_remote);
boost::shared_ptr<transfer> t = m_transfer.lock();
if (!t)
return;
// do not check a hash, due to mldonkey's weirdness
// mldonkey sends zero hash here write_accept_upload();
} else {
ERR("start upload error " << error.message() << " <== " << m_remote);
}
}
void peer_connection::write_accept_upload()
{
DBG("accept upload ==> " << m_remote);
client_accept_upload au;
write_struct(au);
}
當下載方收到client_accept_upload消息的時候就可以向上傳方發送請求文件數據塊的消息了，在libed2k中這條消息的處理邏輯如下：
void peer_connection::on_accept_upload(const error_code& error)
{
if (!error) {
DBG("accept upload <== " << m_remote);
request_block();
send_block_requests();
} else {
ERR("accept upload error " << error.message() << " <== " << m_remote);
}
}
篩選需要下載哪一塊的算法比較複雜，我們不在這一章中討論。上面這個函數做的事情大致是：
- 在缺失的數據分片中按照優先級和策略設定尋找一塊或者多塊對方擁有的文件塊。
- 計算出將這些文件塊的偏移，裝到“請求文件數據塊”的消息中發給上傳方。
4.3.6 "請求文件數據塊"消息以及響應
當文件下載一方收到了對方發出的OP_ACCEPTUPLOADREQ時可以向對方發送請求數據分片的消息OP_REQUESTPARTS(0x47)或者是OP_REQUESTPARTS_I64(0xA3)，兩者差別是內部的偏移值是4個字節表示還是8個字節表示，着意味着使用OP_REQUESTPARTS時只能傳輸4G內的文件。在libed2k中使用的是OP_REQUESTPARTS_I64消息，但是在emule protocol 中只能32位偏移的版本（emule protocol 6.4.4 Request file parts）如下，紅字代表的是OP_REQUESTPARTS_I64對應的值：
Send to a peer client to request file parts. The message may request a max number of 3 file
parts to download.
|
Name |
Size in bytes |
Default Value |
Comment |
|
Protocol |
1 |
0xE3 |
|
|
Size |
4 |
The size of the message in bytes not including the header and size fields |
|
|
Type |
1 |
0x47(0xA3) |
The value of the OP_REQUESTPARTS （OP_REQUESTPARTS_I64）op-code |
|
File ID |
16 |
NA |
A unique file ID calculated by hashing the file’s data |
|
Part1 Start offset |
4（8） |
NA |
Start offset of the part 1 in the file |
|
Part2 Start offset |
4（8） |
NA |
|
|
Part3 Start offset |
4（8） |
NA |
|
|
Part 1 End offset |
4（8） |
NA |
End offset of the part 1 in the file |
|
Part 2 End offset |
4（8） |
NA |
|
|
Part 3 End offset |
4（8） |
NA |
libed2k 中發送這個消息的代碼如下：
void peer_connection::send_block_requests() {
if (m_channel_state[upload_channel] & peer_info::bw_seq)
return; boost::shared_ptr<transfer> t = m_transfer.lock();
if (!t || m_disconnecting)
return;
// we can't download pieces in these states
//if (t->state() == transfer_status::checking_files || ...) return;
// send in 3 requests at a time
if (m_download_queue.size() + 2 >= m_desired_queue_size || t->upload_mode())
return;
client_request_parts_64 rp; rp.m_hFile = t->hash();
while (!m_request_queue.empty() && m_download_queue.size() < m_desired_queue_size) {
pending_block block = m_request_queue.front();
m_request_queue.erase(m_request_queue.begin());
// if we're a seed, we don't have a piece picker so we don't have to
// worry about invariants getting out of sync with it
if (t->is_seed()) continue;
// this can happen if a block times out, is re-requested and
// then arrives "unexpectedly"
if (t->picker().is_finished(block.block) || t->picker().is_downloaded(block.block)) {
t->picker().abort_download(block.block);
continue;
}
m_download_queue.push_back(block);
rp.append(block_range(block.block.piece_index, block.block.block_index, t->size()));
if (rp.full()) {
write_request_parts(rp);
rp.reset();
}
}
if (!rp.empty())
write_request_parts(rp);
}
可以看到通過這裏通過piece_picker對象（t->picker()）選擇分片，然後計算分片的偏移，再調用client_request_parts_64::append加入到client_request_parts_64對象中，最後調用write_request_parts將這個對象發送出去，client_request_parts_64定義如下：
typedef client_request_parts<boost::uint64_t> client_request_parts_64;
template <typename size_type>
struct client_request_parts
{
md4_hash m_hFile;
size_type m_begin_offset[3];
size_type m_end_offset[3];
size_t m_parts;
client_request_parts() { reset(); }
void reset() {
m_parts = 0;
std::memset(m_begin_offset, 0, sizeof(m_begin_offset));
std::memset(m_end_offset, 0, sizeof(m_end_offset));
}
void append(std::pair<size_type, size_type> range) {
assert(!full());
m_begin_offset[m_parts] = range.first;
m_end_offset[m_parts] = range.second;
++m_parts;
}
bool full() const { return m_parts > 2; }
bool empty() const { return m_parts == 0; }
template<typename Archive>
void serialize(Archive& ar) {
ar & m_hFile
& m_begin_offset[0] & m_begin_offset[1] & m_begin_offset[2]
& m_end_offset[0] & m_end_offset[1] & m_end_offset[2];
}
};
OP_REQUESTPARTS/OP_REQUESTPARTS_I64消息的響應：
template <typename Struct>
void peer_connection::on_request_parts(const error_code& error)
{
if (!error) {
boost::shared_ptr<transfer> t = m_transfer.lock();
if (!t)
return;
DECODE_PACKET(Struct, rp);
for (size_t i = 0; i < 3; ++i) {
std::vector<peer_request> reqs = mk_peer_requests(rp.m_begin_offset[i], rp.m_end_offset[i], t->size());
for (std::vector<peer_request>::const_iterator ri = reqs.begin(); ri != reqs.end(); ++ri) {
if (t->have_piece(ri->piece)) {
m_requests.push_back(*ri);
}
else {
DBG("we haven't piece " << ri->piece << " requested from " << m_remote);
}
}
if (reqs.empty()) {
//ERR("incorrect request ["
}
}
fill_send_buffer();
}
else {
ERR("request parts error " << error.message() << " <== " << m_remote);
}
}
上傳方將收到的塊偏移打包到peer_request並添加到m_requests隊列，最後調用fill_send_buffer，實現如下：
void peer_connection::fill_send_buffer()
{
if (m_channel_state[upload_channel] & peer_info::bw_seq)
return;
if (m_handshake_complete) {
send_deferred();
}
if (!m_requests.empty() && m_send_buffer.size() < m_ses.settings().send_buffer_watermark)
{
const peer_request& req = m_requests.front();
write_part(req);
send_data(req);
m_requests.erase(m_requests.begin());
}
}
這個函數從m_requests取出請求，調用write_part和send_data將對應的數據發送出去。write_part實現如下：
void peer_connection::write_part(const peer_request& r)
{
boost::shared_ptr<transfer> t = m_transfer.lock();
if (!t) return;
client_sending_part_64 sp;
std::pair<size_type, size_type> range = mk_range(r);
sp.m_hFile = t->hash();
sp.m_begin_offset = range.first;
sp.m_end_offset = range.second;
write_struct(sp);
}
client_sending_part_64封裝的是OP_SENDINGPART_I64消息，定義如下：
template <typename size_type>
struct client_sending_part
{
md4_hash m_hFile;
size_type m_begin_offset;
size_type m_end_offset;
// user_data[end-begin]
template<typename Archive>
void serialize(Archive& ar){
ar & m_hFile & m_begin_offset & m_end_offset;
// user_data[end-begin]
}
};
在fill_send_buffer中，緊跟着write_part是send_data函數，它的實現如下：
void peer_connection::send_data(const peer_request& req)
{
boost::shared_ptr<transfer> t = m_transfer.lock();
if (!t) return;
std::pair<peer_request, peer_request> reqs = split_request(req);
peer_request r = reqs.first;
peer_request left = reqs.second;
if (r.length > 0)
{
t->filesystem().async_read(r, boost::bind(&peer_connection::on_disk_read_complete,
self_as<peer_connection>(), _1, _2, r, left));
m_channel_state[upload_channel] |= peer_info::bw_seq;
}
else {
m_channel_state[upload_channel] &= ~peer_info::bw_seq;
fill_send_buffer();
}
}
可以看到，上傳方在收到數據傳輸請求後，如果符合條件則發送一條OP_SENDINGPART_I64消息，緊跟其後是和這條消息中設定的一段文件數據，這條消息的細節如下(emule protocol 6.4.3 Sending file part)：
注意協議中只有OP_SENDINGPART，OP_SENDINGPART_I64與之不同的是紅色文字部分。This message contains a part of a downloaded file. The message size (in the header) indicates the overall part size and not only the size of the packet in which this message is sent. This message is divided to several data packets each with payload smaller than the maximum TCP MTU - in eMule 0.30e the payload size is 1300 bytes. Section ??（本文檔4.3.1） discusses sending file parts in detail. See also section 6.5.3 for details about sending compressed file parts.
|
Name |
Size in bytes |
Default Value |
Comment |
|
Protocol |
1 |
0xE3 |
|
|
Size |
4 |
The size of the sent part in bytes not including the header and size fields |
|
|
Type |
1 |
0x46 |
The value of the OP_SENDINGPART（OP_SENDINGPART_I64）opcode |
|
File ID |
16 |
NA |
A unique file ID calculated by hashing the file’s data |
|
Start Pos |
4（8） |
NA |
The start position of the downloaded data |
|
End Pos |
4（8） |
NA |
The end position of the downloaded data |
|
Data |
NA |
NA |
The actual downloaded data. This data may be compressed. |
文件下載方對這條消息的處理：
template <typename Struct>
void peer_connection::on_sending_part(const error_code& error)
{
if (!error)
{
DECODE_PACKET(Struct, sp);
DBG("part " << sp.m_hFile
<< " [" << sp.m_begin_offset << ", " << sp.m_end_offset << "]"
<< " <== " << m_remote);
peer_request r = mk_peer_request(sp.m_begin_offset, sp.m_end_offset);
receive_data(r, false);
}
else
{
ERR("part error " << error.message() << " <== " << m_remote);
}
}
receive_data的實現如下：
void peer_connection::receive_data(const peer_request& req, bool compressed)
{
LIBED2K_ASSERT((m_channel_state[download_channel] & (peer_info::bw_network | peer_info::bw_seq)) == 0);
LIBED2K_ASSERT(req.length <= BLOCK_SIZE);
m_recv_pos = 0;
m_recv_req = req;
m_recv_compressed = compressed;
m_channel_state[download_channel] |= peer_info::bw_seq;
receive_data();
}
以及它的重載：
void peer_connection::receive_data()
{
if (!has_download_bandwidth())
return;
//...
int max_receive = std::min<int>(remained_bytes, m_quota[download_channel]);
if (max_receive > 0){
m_channel_state[download_channel] |= peer_info::bw_network;
boost::asio::async_read(
*m_socket, boost::asio::buffer(b->buffer + offset_in_block(m_recv_req) + m_recv_pos, max_receive),
make_read_handler(boost::bind(&peer_connection::on_receive_data,
self_as<peer_connection>(), _1, _2)));
}else{
do_read();
}
}
在peer_connection::on_receive_data中：
void peer_connection::on_receive_data(const error_code& error, std::size_t bytes_transferred)
{
//...
if (m_recv_pos == m_recv_req.length)
{
if (complete_block(*b))
{
//...
fs.async_write(req, holder,
boost::bind(&peer_connection::on_disk_write_complete,
self_as<peer_connection>(),
_1, _2, req, t));
}
//...
m_channel_state[download_channel] &= ~peer_info::bw_seq;
request_block();
send_block_requests();
}
do_read();
}
可以看到收到數據了以後調用fs.async_write寫入到文件中（還有檢查，校驗等等），然後再調用request_block()和send_block_requests()請求下一塊數據，循環往復直到文件下載完成。