mtcp协议栈
mTCP:A Highly Scalable User-level TCP Stack for Multicore Systems
EunYoung Jeong,Shinae Woo,Muhammad Jamshed,Haewon Jeong
Sunghwan Ihm*,Dongsu Han,and KyoungSoo Park
KAIST*Princeton University
Abstract
Scaling the performance of short TCP connections on multicore systems is fundamentally challenging.Although many proposals have attempted to address various short-comings,inef?ciency of the kernel implementation still persists.For example,even state-of-the-art designs spend 70%to80%of CPU cycles in handling TCP connections in the kernel,leaving only small room for innovation in the user-level program.
This work presents mTCP,a high-performance user-level TCP stack for multicore systems.mTCP addresses the inef?ciencies from the ground up—from packet I/O and TCP connection management to the application inter-face.In addition to adopting well-known techniques,our design(1)translates multiple expensive system calls into a single shared memory reference,(2)allows ef?cient?ow-level event aggregation,and(3)performs batched packet I/O for high I/O ef?ciency.Our evaluations on an8-core machine showed that mTCP improves the performance of small message transactions by a factor of25compared to the latest Linux TCP stack and a factor of3compared to the best-performing research system known so far.It also improves the performance of various popular applications by33%to320%compared to those on the Linux stack. 1Introduction
Short TCP connections are becoming widespread.While large content transfers(e.g.,high-resolution videos)con-sume the most bandwidth,short“transactions”1dominate the number of TCP?ows.In a large cellular network,for example,over90%of TCP?ows are smaller than32KB and more than half are less than4KB[45].
Scaling the processing speed of these short connec-tions is important not only for popular user-facing on-line services[1,2,18]that process small messages.It is 1We refer to a request-response pair as a transaction.These transac-tions are typically small in size.also critical for backend systems(e.g.,memcached clus-ters[36])and middleboxes(e.g.,SSL proxies[32]and redundancy elimination[31])that must process TCP con-nections at high speed.Despite recent advances in soft-ware packet processing[4,7,21,27,39],supporting high TCP transaction rates remains very challenging.For exam-ple,Linux TCP transaction rates peak at about0.3million transactions per second(shown in Section5),whereas packet I/O can scale up to tens of millions packets per second[4,27,39].
Prior studies attribute the inef?ciency to either the high system call overhead of the operating system[28,40,43] or inef?cient implementations that cause resource con-tention on multicore systems[37].The former approach drastically changes the I/O abstraction(e.g.,socket API) to amortize the cost of system calls.The practical lim-itation of such an approach,however,is that it requires signi?cant modi?cations within the kernel and forces ex-isting applications to be re-written.The latter one typically makes incremental changes in existing implementations and,thus,falls short in fully addressing the inef?ciencies. In this paper,we explore an alternative approach that de-livers high performance without requiring drastic changes to the existing code base.In particular,we take a clean-slate approach to assess the performance of an untethered design that divorces the limitation of the kernel implemen-tation.To this end,we build a user-level TCP stack from the ground up by leveraging high-performance packet I/O libraries that allow applications to directly access the packets.Our user-level stack,mTCP,is designed for three explicit goals:
1.Multicore scalability of the TCP stack.
2.Ease of use(i.e.,application portability to mTCP).
3.Ease of deployment(i.e.,no kernel modi?cations). Implementing TCP in the user level provides many opportunities.In particular,it can eliminate the expen-sive system call overhead by translating syscalls into inter-process communication(IPC).However,it also in-
Accept queue
Conn.Locality Socket API Event Handling Packet I/O
Application Mod-i?cation Kernel
Modi?cation PSIO [12],No TCP stack Batched
DPDK [4],No interface for No
PF RING [7],transport layer (NIC driver)netmap [21]Linux-2.6Shared None BSD socket Syscalls Per packet Transparent No Linux-3.9Per-core None BSD socket Syscalls Per packet Add option
No SO REUSEPORT
Af?nity-Accept [37]Per-core Yes BSD socket Syscalls Per packet Transparent Yes MegaPipe [28]Per-core Yes lwsocket Batched syscalls Per packet Event model to Yes completion I/O FlexSC [40],Shared None BSD socket Batched syscalls Per packet Change to use Yes VOS [43]new API
mTCP
Per-core
Yes
User-level socket
Batched function calls
Batched
Socket API to mTCP API
No
(NIC driver)
Table 1:Comparison of the bene?ts of previous work and mTCP.
troduces fundamental challenges that must be addressed—processing IPC messages,including shared memory mes-sages,involve context-switches that are typically much more expensive than the system calls themselves [3,29].Our key approach is to amortize the context-switch overhead over a batch of packet-level and socket-level events.While packet-level batching [27]and system-call batching [28,40,43](including socket-level events)have been explored individually,integrating the two requires a careful design of the networking stack that translates packet-level events to socket-level events and vice-versa.This paper makes two key contributions:
First,we demonstrate that signi?cant performance gain can be obtained by integrating packet-and socket-level batching.In addition,we incorporate all known optimizations,such as per-core listen sockets and load balancing of concurrent ?ows on multicore CPUs with receive-side scaling (RSS).The resulting TCP stack out-performs Linux and MegaPipe [28]by up to 25x (w/o SO_REUSEPORT )and 3x,respectively,in handling TCP transactions.This directly translates to application per-formance;mTCP increases existing applications’perfor-mance by 33%(SSLShader)to 320%(lighttpd).
Second,unlike other designs [23,30],we show that such integration can be done purely at the user level in a way that ensures ease of porting without requiring signi?cant modi?cations to the kernel.mTCP provides BSD-like socket and epoll-like event-driven interfaces.Migrating existing event-driven applications is easy since one simply needs to replace the socket calls to their counterparts in mTCP (e.g.,accept()becomes mtcp_accept())and use the per-core listen socket.
2Background and Motivation
We ?rst review the major inef?ciencies in existing TCP implementations and proposed solutions.We then discuss our motivation towards a user-level TCP stack.
2.1Limitations of the Kernel’s TCP Stack
Recent studies proposed various solutions to address four major inef?ciencies in the Linux TCP stack:lack of con-nection locality,shared ?le descriptor space,inef?cient packet processing,and heavy system call overhead [28].Lack of connection locality:Many applications are multi-threaded to scale their performance on multicore systems.However,they typically share a listen socket that accepts incoming connections on a well-known port.As a result,multiple threads contend for a lock to access the socket’s accept queue,resulting in a signi?cant perfor-mance degradation.Also,the core that executes the kernel code for handling a TCP connection may be different from the one that runs the application code that actually sends and receives data.Such lack of connection locality intro-duces additional overhead due to increased CPU cache misses and cache-line sharing [37].
Af?nity-Accept [37]and MegaPipe [28]address this issue by providing a local accept queue in each CPU core and ensuring ?ow-level core af?nity across the kernel and application thread.Recent Linux kernel (3.9.4)also partly addresses this by introducing the SO_REUSEPORT [14]op-tion,which allows multiple threads/processes to bind to the same port number.
Shared ?le descriptor space:In POSIX-compliant op-erating systems,the ?le descriptor (fd)space is shared within a process.For example,Linux searches for the min-imum available fd number when allocating a new socket.In a busy server that handles a large number of concurrent connections,this incurs signi?cant overhead due to lock contention between multiple threads [20].The use of ?le descriptors for sockets,in turn,creates extra overhead of going through the Linux Virtual File System (VFS),a pseudo-?lesystem layer for supporting common ?le op-erations.MegaPipe eliminates this layer for sockets by explicitly partitioning the fd space for sockets and regular ?les [28].
libraries[4,7,27,39]address these problems,these li-braries do not provide a full-?edged TCP stack,and not all optimizations are incorporated into the kernel. System call overhead:The BSD socket API requires frequent user/kernel mode switching when there are many short-lived concurrent connections.As shown in FlexSC[40]and VOS[43],frequent system calls can result in processor state(e.g.,top-level caches,branch prediction table,etc.)pollution that causes performance penalties.Previous solutions propose system call batch-ing[28,43]or ef?cient system call scheduling[40]to amortize the cost.However,it is dif?cult to readily apply either approach to existing applications since they often require user and/or kernel code modi?cation due to the changes to the system call interface and/or its semantics. Table1summarizes the bene?ts provided by previous work compared to a vanilla Linux kernel.Note that there is not a single system that provides all of the bene?ts. 2.2Why User-level TCP?
While many previous designs have tried to scale the per-formance of TCP in multicore systems,few of them truly overcame the aforementioned inef?ciencies of the kernel. This is evidenced by the fact that even the best-performing system,MegaPipe,spends a dominant portion of CPU cycles(～80%)inside the kernel.Even more alarming is the fact that these CPU cycles are not utilized ef?ciently; according to our own measurements,Linux spends more than4x the cycles(in the kernel and the TCP stack com-bined)than mTCP does while handling the same number of TCP transactions.
To reveal the signi?cance of this problem,we pro?le the server’s CPU usage when it is handling a large number of concurrent TCP transactions(8K to48K concurrent TCP connections).For this experiment,we use a simple web server(lighttpd v1.4.32[8])running on an8-core Intel
cycle in the kernel(including TCP/IP and I/O)across four lighttpd versions.
Xeon CPU(2.90GHz,E5-2690)with32GB of memory and a10Gbps NIC(Intel82599chipsets).Our clients use ab v2.3[15]to repeatedly download a64B?le per connection.Multiple clients are used in our experiment to saturate the CPU utilization of the server.Figure1shows the breakdown of CPU usage comparing four versions of the lighttpd server:a multithreaded version that harnesses all8CPU cores on Linux2.6.32and3.10.122(Linux),a version ported to MegaPipe3(MegaPipe),and a version using mTCP,our user-level TCP stack,on Linux2.6.32 (mTCP).Note that MegaPipe adopts all recent optimiza-tions such as per-core accept queues and?le descriptor space,as well as user-level system call batching,but reuses the existing kernel for packet I/O and TCP/IP processing. Our results indicate that Linux and MegaPipe spend 80%to83%of CPU cycles in the kernel which leaves only a small portion of the CPU to user-level applications. Upon further investigation,we?nd that lock contention for shared in-kernel data structures,buffer management, and frequent mode switch are the main culprits.This implies that the kernel,including its stack,is the major bottleneck.Furthermore,the results in Figure2show that the CPU cycles are not spent ef?ciently in Linux and MegaPipe.The bars indicate the relative number of transactions processed per each CPU cycle inside the kernel and the TCP stack(e.g.,outside the application), normalized by the performance of Linux2.6.32.We?nd that mTCP uses the CPU cycles4.3times more effectively than Linux.As a result,mTCP achieves3.1x and1.8x the performance of Linux2.6and MegaPipe,respectively, while using fewer CPU cycles in the kernel and the TCP stack.
Now,the motivation of our work is clear.Can we de-sign a user-level TCP stack that incorporates all existing optimizations into a single system and achieve all bene?ts that individual systems have provided in the past?How much of a performance improvement can we get if we build such a system?Can we bring the performance of existing packet I/O libraries to the TCP stack?
2This is the latest Linux kernel version as of this writing.
3We use Linux3.1.3for MegaPipe due to its patch availability.
the user https://www.wendangku.net/doc/154366542.html,er-level TCP is attractive for many rea-sons.First,it allows us to easily depart from the kernel’s complexity.In particular,due to shared data structures and various semantics that the kernel has to support(e.g., POSIX and VFS),it is often dif?cult to separate the TCP stack from the rest of the kernel.Furthermore,it allows us to directly take advantage of the existing optimiza-tions in the high-performance packet I/O library,such as netmap[39]and Intel DPDK[4].Second,it allows us to apply batch processing as the?rst principle,harnessing the ideas in FlexSC[40]and VOS[43]without extensive kernel modi?cations.In addition to performing batched packet I/O,the user-level TCP naturally collects multiple ?ow-level events to and from the user application(e.g., connect()/accept()and read()/write()for differ-ent connections)without the overhead of frequent mode switching in system calls.Finally,it allows us to easily preserve the existing application programming interface. Our TCP stack is backward-compatible in that we provide a BSD-like socket interface.
3Design
The goal of mTCP is to achieve high scalability on mul-ticore systems while maintaining backward compatibil-ity to existing multi-threaded,event-driven applications. Figure3presents an overview of our system.At the high-est level,applications link to the mTCP library,which provides a socket API and an event-driven programming interface for backward compatibility.The two underlying components,user-level TCP stack and packet I/O library, are responsible for achieving high scalability.Our user-level TCP implementation runs as a thread on each CPU core within the same application process.The mTCP thread directly transmits and receives packets to and from the NIC using our custom packet I/O library.Existing user-level packet libraries only allow one application to access an NIC port.Thus,mTCP can only support one application per NIC port.However,we believe this can be addressed in the future using virtualized network inter-faces(more details in Section3.3).Applications can still
the existing TCP stack,provided that
are not used by mTCP.
?rst present the design of mTCP’s
components in Sections3.1
the API and semantics that
applications in Section3.3.
Packet I/O Library
allow high-speed packet I/O
from a user-level application[4,7,
are not suitable for implementing a
their interface is mainly based on
signi?cantly waste precious CPU cy-cles that can potentially bene?t the applications.Further-more,our system requires ef?cient multiplexing between TX and RX queues from multiple NICs.For example,we do not want to block a TX queue while sending a data packet when a control packet is waiting to be received. This is because if we block the TX queue,important con-trol packets,such as SYN or ACK,may be dropped,re-sulting in a signi?cant performance degradation due to retransmissions.
To address these challenges,mTCP extends the Pack-etShader I/O engine(PSIO)[27]to support an ef?cient event-driven packet I/O interface.PSIO offers high-speed packet I/O by utilizing RSS that distributes incoming pack-ets from multiple RX queues by their?ows,and provides ?ow-level core af?nity to minimize the contention among the CPU cores.On top of PSIO’s high-speed packet I/O, the new event-driven interface allows an mTCP thread to ef?ciently wait for events from RX and TX queues from multiple NIC ports at a time.
The new event-driven interface,ps_select(),works similarly to select()except that it operates on TX/RX queues of interested NIC ports for packet I/O.For exam-ple,mTCP speci?es the interested NIC interfaces for RX and/or TX events with a timeout in microseconds,and ps_select()returns immediately if any event of interest is available.If such an event is not detected,it enables the interrupts for the RX and/or TX queues and yields the thread context.Eventually,the interrupt handler in the driver wakes up the thread if an I/O event becomes available or the timeout expires.ps_select()is also similar to the select()/poll()interface supported by netmap[39].However,unlike netmap,we do not integrate this with the general-purpose event system in Linux to avoid its overhead.
The use of PSIO brings the opportunity to amortize the overhead of system calls and context switches throughout the entire system,in addition to eliminating the per-packet memory allocation and DMA overhead.In PSIO,packets are received and transmitted in batches[27],amortizing the cost of expensive PCIe operations,such as DMA ad-dress mapping and IOMMU lookups.
thread.This“zero-thread TCP”could potentially provide the best performance since this translates costly system calls into light-weight user-level function calls.However, the fundamental limitation of this approach is that the correctness of internal TCP processing depends on the timely invocation of TCP functions from the application. In mTCP,we choose to create a separate TCP thread to avoid such an issue and to minimize the porting effort for existing applications.Figure4shows how mTCP interacts with the application thread.The application uses mTCP library functions that communicate with the mTCP thread via shared buffers.The access to the shared buffers is granted only through the library functions,which allows safe sharing of the internal TCP data.When a library function needs to modify the shared data,it simply places a request(e.g.,write()request)to a job queue.This way, multiple requests from different?ows can be piled to the job queue at each loop,which are processed in batch when the mTCP thread regains the CPU.Flow events from the mTCP thread(e.g.,new the CPU core.Flow events from the mTCP thread(e.g.,new connections,new data arrival, etc.)are delivered in a similar way
This,however,requires additional overhead of manag-ing concurrent data structures and context switch between the application and the mTCP thread.Such cost is un-fortunately not negligible,typically much larger than the system call overhead[29].One measurement on a recent Intel CPU shows that a thread context switch takes19 times the duration of a null system call[3].
In this section,we describe how mTCP addresses these challenges and achieves high scalability with the user-level TCP stack.We?rst start from how mTCP processes TCP packets in Section3.2.1,then present a set of key optimizations we employ to enhance its performance in Sections3.2.2,3.2.3,and3.2.4.
3.2.1Basic TCP Processing
When the mTCP thread reads a batch of packets from the NIC’s RX queue,mTCP passes them to the TCP packet ?ow hash table.As in Figure5,if a server side receives an ACK for its SYN/ACK packet(1),the tcb for the new connection will be enqueued to an accept queue(2),and a read event is generated for the listening socket(3).If a new data packet arrives,mTCP copies the payload to the socket’s read buffer and enqueues a read event to an internal event queue.mTCP also generates an ACK packet and keeps it in the ACK list of a TX manager until it is written to a local TX queue.
After processing a batch of received packets,mTCP ?ushes the queued events to the application event queue (4)and wakes up the application by signaling it.When the application wakes up,it processes multiple events in a single event loop(5),and writes responses from multiple ?ows without a context switch.Each socket’s write() call writes data to its send buffer(6),and enqueues its tcb to the write queue(7).Later,mTCP collects the tcb s that have data to send,and puts them into a send list(8). Finally,a batch of outgoing packets from the list will be sent by a packet I/O system call,transmitting them to the NIC’s TX queue.
3.2.2Lock-free,Per-core Data Structures
To minimize inter-core contention between the mTCP threads,we localize all resources(e.g.,?ow pool,socket buffers,etc.)in each core,in addition to using RSS for ?ow-level core af?nity.Moreover,we completely elimi-nate locks by using lock-free data structures between the application and mTCP.On top of that,we also devise an ef?cient way of managing TCP timer operations. Thread mapping and?ow-level core af?nity:We pre-serve?ow-level core af?nity in two stages.First,the packet I/O layer ensures to evenly distribute TCP con-nection workloads across available CPU cores with RSS. This essentially reduces the TCP scalability problem to each core.Second,mTCP spawns one TCP thread for each application thread and co-locates them in the same physical CPU core.This preserves the core af?nity of
packet and?ow processing,while allowing them to
the same CPU cache without cache-line sharing. Multi-core and cache-friendly data structures: keep most data structures,such as the?ow hash socket id manager,and the pool of tcb and socket local to each TCP thread.This signi?cantly reduces sharing across threads and CPU cores,and achieves parallelism.When a data structure must be shared threads(e.g.,between mTCP and the application
we keep all data structures local to each core and lock-free data structures by using a single-producer single-consumer queue.We maintain write,connect, close queues,whose requests go from the mTCP,and an accept queue where new connections delivered from mTCP to the application.
In addition,we keep the size of frequently
data structures small to maximize the bene?t of the CPU cache,and make them aligned with the size of a CPU cache line to prevent any false sharing.For example,we divide tcb into two parts where the?rst-level structure holds64bytes of the most frequently-accessed?elds and two pointers to next-level structures that have128and192 bytes of receive/send-related variables,respectively. Lastly,to minimize the overhead of frequent memory allocation/deallocation,we allocate a per-core memory pool for tcb s and socket buffers.We also utilize huge pages to reduce the TLB misses when accessing the tcb s. Because their access pattern is essentially random,it often causes a large number of TLB misses.Putting the memory pool of tcb s and a hash table that indexes them into huge pages reduces the number of TLB misses.
Ef?cient TCP timer management:TCP requires timer operations for retransmission timeouts,connections in the TIME WAIT state,and connection keep-alive checks. mTCP provides two types of timers:one managed by a sorted list and another built with a hash table.For coarse-grained timers,such as managing connections in the TIME W AIT state and connection keep-alive check, we keep a list of tcb s sorted by their timeout values.Ev-ery second,we check the list and handle any tcb s whose timers have expired.Note that keeping the list sorted is trivial since a newly-added entry should have a strictly larger timeout than any of those that are already in the list.For?ne-grained retransmission timers,we use the remaining time(in milliseconds)as the hash table index, and process all tcb s in the same bucket when a timeout ex-pires for the bucket.Since retransmission timers are used by virtually all tcb s whenever a data(or SYN/FIN)packet is sent,keeping a sorted list would consume a signi?cant amount of CPU cycles.Such?ne-grained event batch processing with millisecond granularity greatly reduces the overhead.ets in batch,mTCP processes them to generate a batch of?ow-level events.These events are then passed up to the application,as illustrated in Figure6.The TX direc-tion works similarly,as the mTCP library transparently batches the write events into a write queue.While the idea of amortizing the system call overhead using batches is not new[28,43],we demonstrate that bene?ts similar to that of batched syscalls can be effectively achieved in user-level TCP.
In our experiments with8RX/TX queues per10Gbps port,the average number of events that an mTCP thread generates in a single scheduling period is about2,170 for both TX and RX directions(see Section5.1).This ensures that the cost of a context switch is amortized over a large number of events.Note the fact that the use of multiple queues does not decrease the number of the events processed in a batch.
3.2.4Optimizing for Short-lived Connections
We employ two optimizations for supporting many short-lived concurrent connections.
Priority-based packet queueing:For short TCP con-nections,the control packets(e.g.,SYN and FIN)have a critical impact on the performance.Since the control pack-ets are mostly small-sized,they can often be delayed for a while when they contend for an output port with a large number of data packets.We prioritize control packets by keeping them in a separate list.We maintain three kinds of lists for TX as shown in Figure5.First,a control list contains the packets that are directly related to the state of a connection such as SYN,SYN/ACK,and ACK,or FIN and FIN/ACK.We then manage ACKs for incoming data packets in an ACK list.Finally,we keep a data list to send data in the socket buffers of TCP?ows.When we put actual packets in a TX queue,we?rst?ll the packets from a control list and an ACK list,and later queue the data packets.By doing this,we prioritize important packets
to prevent short connections from being delayed by other long connections.4
Lightweight connection setup:In addition,we?nd that a large portion of connection setup cost is from allo-cating memory space for TCP control blocks and socket buffers.When many threads concurrently call malloc() or free(),the memory manager in the kernel can be eas-ily contended.To avoid this problem,we pre-allocate large memory pools and manage them at user level to sat-isfy memory(de)allocation requests locally in the same thread.
3.3Application Programming Interface One of our primary design goals is to minimize the port-ing effort of existing applications so that they can easily bene?t from our user-level TCP stack.Therefore,our programming interface must preserve the most commonly used semantics and application interfaces as much as pos-sible.To this end,mTCP provides a socket API and an event-driven programming interface.
User-level socket API:We provide a BSD-like socket interface;for each BSD socket function,we have a corresponding function call(e.g.,accept()becomes mtcp_accept()).In addition,we provide functionali-ties that are frequently used with sockets,e.g.,fcntl and ioctl,for setting the socket as nonblocking or getting/set-ting the socket buffer size.To support various applications that require inter-process communication using pipe(), we also provide mtcp_pipe().
The socket descriptor space in mTCP(including the fds of pipe()and epoll())is local to each mTCP thread; each mTCP socket is associated with a thread context. This allows parallel socket creation from multiple threads by removing lock contention on the socket descriptor space.We also relax the semantics of socket()such that it returns any available socket descriptor instead of the minimum available fd.This reduces the overhead of ?nding the minimum available fd.
User-level event system:We provide an epoll()-like event system.While our event system aggre-gates the events from multiple?ows for batching ef-fects,we do not require any modi?cation in the event handling logic.Applications can fetch the events through mtcp_epoll_wait()and register events through mtcp_epoll_ctl(),which correspond to epoll_wait() and epoll_ctl()in Linux.Our current mtcp_epoll() implementation supports events from mTCP sockets(in-cluding listening sockets)and pipes.We plan to integrate other types of events(e.g.,timers)in the future.
4This optimization can potentially make the system more vulnerable to attacks,such as SYN?ooding.However,existing solutions,such as SYN cookies,can be used to mitigate the problem.Applications:mTCP integrates all techniques known at the time of this writing without requiring substantial kernel modi?cation while preserving the application inter-face.Thus,it allows applications to easily scale their performance without modifying their logic.We have ported many applications,including lighttpd,ab,and SSLShader to use mTCP.For most applications we ported, the number of lines changed were less than100(more de-tails in Section4).We also demonstrate in Section5that a variety of applications can directly enjoy the performance bene?t by using mTCP.
However,this comes with a few trade-offs that appli-cations must consider.First,the use of shared memory space offers limited protection between the TCP stack and the application.While the application cannot directly ac-cess the shared buffers,bugs in the application can corrupt the TCP stack,which may result in an incorrect behavior. Although this may make debugging more dif?cult,we believe this form of fate-sharing is acceptable since users face a similar issue in using other shared libraries such as dynamic memory allocation/deallocation.Second,appli-cations that rely on the existing socket fd semantics must change their logic.However,most applications rarely de-pend on the minimum available fd at socket(),and even if so,porting them will not require signi?cant code change. Third,moving the TCP stack will also bypass all existing kernel services,such as the?rewall and packet schedul-ing.However,these services can also be moved into the user-level and provided as application modules.Finally, our prototype currently only supports a single application due to the limitation of the user-level packet I/O system. We believe,however,that this is not a fundamental limita-tion of our approach;hardware-based isolation techniques such as VMDq[5]and SR-IOV[13]support multiple virtual guest stacks inside the same host using multiple RX/TX queues and hardware-based packet classi?cation. We believe such techniques can be leveraged to support multiple applications that share a NIC port.
4Implementation
We implement11,473lines of C code(LoC),including packet I/O,TCP?ow management,user-level socket API and event system,and552lines of code to patch the PSIO library.5For threading and thread synchronization,we use pthread,the standard POSIX thread library[11].
Our TCP implementation follows RFC793[17].It sup-ports basic TCP features such as connection management, reliable data transfer,?ow control,and congestion control. For reliable transfer,it implements cumulative acknowl-edgment,retransmission timeout,and fast retransmission. mTCP also implements popular options such as timestamp, Maximum Segment Size(MSS),and window scaling.For 5The number is counted by SLOCCount2.26.
congestion control,mTCP implements NewReno[10], but it can easily support other mechanisms like TCP CU-BIC[26].For correctness,we have extensively tested our mTCP stack against various versions of Linux TCP stack, and have it pass stress tests,including cases where a large number of packets are lost or reordered.
4.1mTCP Socket API
Our BSD-like socket API takes on per-thread semantics. Each mTCP socket function is required to have a context, mctx_t,which identi?es the corresponding mTCP thread. Our event noti?cation function,mtcp_epoll,also enables easy migration of existing event-driven applications.List-ing1shows an example mTCP application.
mctx_t mctx=mtcp_create_context();
ep_id=mtcp_epoll_create(mctx,N);
mtcp_listen(mctx,listen_id,4096);
while(1){
n=mtcp_epoll_wait(mctx,ep_id,events,N,-1);
for(i=0;i sockid=events[i].data.sockid; if(sockid==listen_id){ c=mtcp_accept(mctx,listen_id,NULL); mtcp_setsock_nonblock(mctx,c); ev.events=EPOLLIN|EPOLLOUT; ev.data.sockid=c; mtcp_epoll_ctl(mctx,ep_id, EPOLL_CTL_ADD,c,&ev); }else if(events[i].events==EPOLLIN){ r=mtcp_read(mctx,sockid,buf,LEN); if(r==0) mtcp_close(mctx,sockid); }else if(events[i].events==EPOLLOUT){ mtcp_write(mctx,sockid,buf,len); } } } Listing1:Sample mTCP application. mTCP supports mtcp_getsockopt()and mtcp_setsockopt()for socket options,and mtcp_readv()and mtcp_writev()for scatter-gather I/O as well. 4.2Porting Existing Applications We ported four different applications to mTCP. Web server(lighttpd-1.4.32):Lighttpd is an open-sourced single-threaded web server that uses event-driven I/O for servicing client requests.We enabled multi-threading to support a per-core listen socket and ported it to mTCP.We changed only～65LoC to use mTCP-speci?c event and socket function calls.For multi-threading,a total of～800lines6were modi?ed out of lighttpd’s～40,000LoC. We also ported lighttpd to MegaPipe for comparison. Because its API is based on the I/O completion model, 6Some global variables had to be localized to avoid race conditions.the porting required more effort as it involved revamping lighttpd’s event-based fdevent backend library;an ad-ditional126LoC were required to enable MegaPipe I/O from the multi-threaded version. Apache benchmarking tool(ab-2.3):ab is a perfor-mance benchmarking tool that generates HTTP requests. It acts as a client to measure the performance of a Web server.Scaling its performance is important because sat-urating a10Gbps port with small transactions requires multiple machines that run ab.However,with mTCP we can reduce the number of machines by more than a factor of4(see Section5.3). Porting ab was similar to porting lighttpd since ab is also single-threaded.However,ab uses the Apache Portable Runtime(APR)library[16]that encapsulates socket func-tion calls,so we ported the APR library(version1.4.6)to use mTCP.We modi?ed29lines of the APR library(out of66,493LoC),and503lines out of2,319LoC of the ab code for making it multi-threaded. SSL reverse proxy(SSLShader):SSLShader is a high-performance SSL reverse proxy that of?oads crypto opera-tions to GPUs[32].For small-?le workloads,SSLShader reports the performance bottleneck in TCP,spending over 60%CPU cycles in the TCP stack,under-utilizing the GPU.Porting SSLShader to mTCP was straightforward since SSLShader was already multi-threaded and uses epoll()for event noti?cation.Besides porting socket function calls,we also replace pipe()with mtcp_pipe(), which is used to notify the completion of crypto operations by GPU threads.Out of6,618lines of C++code,only43 lines were modi?ed to use mTCP.It took less than a day to port to mTCP and to?nish basic testing and debugging. Realistic HTTP replay client/server(WebReplay): WebReplay is a pair of client and server programs that reproduces realistic HTTP traf?c based on the traf?c log collected at a10Gbps backhaul link in a large cellular network[45].Each line in the log has a request URL, a response size,start and end timestamps,and a list of SHA1hashes of the4KB content chunks of the original response.The client generates HTTP requests on start https://www.wendangku.net/doc/154366542.html,ing the content hashes,the server dynami-cally generates a response that preserves the redundancy in the original traf?c;the purpose of the system is to repro-duce Web traf?c with a similar amount of redundancy as the https://www.wendangku.net/doc/154366542.html,ing this,one can test the correctness and performance of network redundancy elimination(NRE) systems that sit between the server and the client.To sim-ulate the traf?c at a high speed,however,the WebReplay server must handle100Ks of concurrent short connections, which requires high TCP performance. WebReplay is multi-threaded and uses the libevent library[6]which in turn calls epoll()for event noti?ca-tion.Porting it to mTCP was mostly straightforward in that it only required replacing the socket and libevent calls with the corresponding mTCP API.We modi?ed44/37 LoC out of1,703/1,663lines of server and client code, respectively. 5Evaluation We answer three questions in this section: 1.Handling short TCP transactions:Does mTCP pro- vide high-performance in handling short transactions? In Section5.1,we show that mTCP outperforms MegaPipe and Linux(w/o SO_REUSEPORT)by3x and 25x,respectively;mTCP connection establishment alone is13x and5x faster than Linux and MegaPipe, respectively. 2.Correctness:Does mTCP provide correctness with- out introducing undesirable side-effects?Section5.2 shows that mTCP provide fairness and does not intro-duce long latency. 3.Application performance:Does mTCP bene?t real applications under realistic workloads?In Section5.3, we show that mTCP increases the performance of various applications running realistic workload by33% (ver- as one GB up per connection,unless otherwise speci?ed.The perfor-mance result is averaged over a one minute period in each experiment.Figure7shows the performance as a function of the number of CPU cores,the number of messages per connection(MPC),and message size. Figure7(a)shows that mTCP scales almost lin-early with the number of CPU cores.Linux without SO_REUSEPORT(‘Linux’)shows poor scaling due to the shared accept queue,and Linux with SO_REUSEPORT (‘REUSEPORT’)scales but not linearly with the num-ber of cores.At8cores,mTCP shows25x,5x,3x higher performance over Linux,REUSEPORT,and MegaPipe, respectively. Figure7(b)shows that the mTCP’s bene?t still holds even when persistent connections are used.mTCP scales well as the number of messages per connection(MPC) increases,and it nearly saturates the10G link from64 MPC.However,the performance of the other systems almost?attens out well below the link capacity.Even at 32MPC,mTCP outperforms all others by a signi?cant margin(up to2.7x),demonstrating mTCP’s effectiveness in handling small packets. Finally,Figure7(c)shows the throughput by varying the message size.mTCP’s performance improvement is more noticeable with small messages,due to its fast processing of small packets.However,both Linux servers fail to saturate the10Gbps link for any message size.MegaPipe saturates the link from4KiB,and mTCP can saturate the link from1KiB messages. Connection accept connection throughputs of a a REUSEPORT,and Multiprocess,respectively.Among the Linux servers,the multi-process version scales the best while other versions show a sudden performance drop at multiple cores.This is due to the contention on the shared accept queue as well as shared fd space.However,Mul-tiprocess shows limited scaling,due to the lack of batch processing and other inef?ciencies in the kernel. 5.2Fairness and Latency Fairness:To verify the throughput fairness among mTCP connections,we use ab to generate8K concurrent connections,each downloading a10MiB?le to saturate a 10Gbps link.On the server side,we run lighttpd with mTCP and Linux TCP.We calculate Jain’s Fairness Index with the(average)transfer rate of each connection.As the value gets closer to 1.0,it shows better fairness.We?nd that Linux and mTCP show0.973and0.999,respectively. mTCP effectively removes the long tail in the response time distribution,whereas Linux often drops SYN packets and enters a long timeout. Latency:Since mTCP relies heavily on batching,one might think it may introduce undesirably long latency. Table2shows the latency breakdown when we run ab with8K concurrent connections against the64B message server.We generate10million requests in total.Linux and mTCP versions respectively achieve45K and428K transactions per second on average.As shown in the table, mTCP slightly increases the minimum(9ms vs.0ms) and the median(13ms vs.3ms)response times.However, the mean and maximum response times are8.8x and54.2x smaller than those of Linux,while handling9.5x more transactions/sec.In addition,the standard deviation of the response times in mTCP is much smaller,implying that mTCP produces more predictable response times,which is becoming increasingly important for modern datacenter applications[33].?le workload from SpecWeb2009. cores.The?le size is64B and8K concurrent connections are MegaPipe,and Linux with and without SO_REUSEPORT. Figure9shows that mTCP improves the throughput of lighttpd by3.2x,2.2x,1.5x over Linux,REUSEPORT, and MegaPipe,respectively.Even though the workload ?ts into the memory,we?nd that heavy system calls for VFS operations limit the performance. We now show the performance of ab.Figure10shows the performance of Linux-based and mTCP-based ab when varying the number of CPU cores when fetching a 64byte?le over HTTP.The scalability of Linux is limited, since it shares the fd space across multiple threads. Figure10shows the performance of ab and the corre-sponding CPU utilization when varying the?le size from 64bytes to8KiB.From2KiB,mTCP saturates the link. saves CPU (not more CPU utilization of SSLShader as a reverse proxy to handle HTTPS trans-actions.SSLShader receives an HTTPS request from ab and decrypts the request.It then fetches the content from lighttpd in plaintext,encrypts the response using SSL,and sends it back to the client.We use 1024-bit RSA,128bit-AES,and HMAC-SHA1as the cipher suite,which is widely used in practice.To measure the performance of SSL handshakes,we have ab to fetch 1-byte objects through SSLShader while varying the number of concur-rent connections. Figure 12shows that mTCP improves the performance over the Linux version by 18%to 33%.As the concur-rency increases,the bene?t of mTCP grows,since mTCP scales better with a large number of concurrent connec-tions.Figure 13indicates that mTCP also reduces the response times compared to the Linux version.Especially,mTCP reduces the tail in the response time distribution over large concurrent connections with a smaller variance,as is also shown in Section 5.2. WebReplay:We demonstrate that mTCP improves the performance of a real HTTP traf?c replayer.We focus on the server’s performance improvement because it performs more interesting work than the client.To fully utilize the server,we use four 10Gbps ports and connect each port a client.The workload (HTTP requests)generated by the on Linux and mTCP stacks.We use 8K concurrent connections in the left graph,and mark median and 95th-percentile numbers. #of copies 1234567Linux (ms)27.829.045.81175.1---mTCP (ms) 0.5 0.9 2.6 8.1 17.5 37.1 79.8 Table 3:Averages of extra delays (in ms)from the original response times when replaying n copies of the log concurrently. #of concurrent #of new connections Bandwidth connections per second (Gbps)Mean 23,86914,425 2.28Min 20,60812,755 1.79Max 25,734 15,440 3.48 Table 4:Log statistics for WebReplay. backhaul link [45].We replay the log for three minutes at a peak time (at 11pm on July 7,2012)during the mea-surement period.The total number of requests within the timeframe is 2.8million with the median and average con-tent size as 1.7KB and 40KB.Table 4summarizes the workload that we replay.Unfortunately,we note that the trace we replay does not simulate the original traf?c per-fectly since a longer log is required to effectively simulate idle connections.Actually,the original traf?c had as much as 270K concurrent connections with more than 1million TCP connections created per minute.To simulate such a load,we run multiple copies of the same log concurrently for this experiment. Table 3compares the averages of extra delays from the original response times when we replay n copies of the log concurrently with Linux and mTCP-based WebRe-player.We ?nd that the Linux server works ?ne up to the concurrent run of three copies,but the average extra delay goes up beyond 1second at four copies.In contrast,mTCP server ?nishes up to seven copies while keeping the average extra delay under 100ms.The main cause for the delay in?ation in the Linux version is the increased number of concurrent TCP transactions,which draws the bottleneck in the TCP stack. 6Related Work We brie?y discuss previous work related to mTCP. System call and I/O batching:Frequent system calls performance bottleneck in busy servers. FlexSC[40]identi?es that CPU cache pollution can waste more CPU cycles than the user/kernel mode switch itself. They batch the system calls by having user and kernel space share the syscall pages,which allows signi?cant performance improvement for event-driven servers[41]. MegaPipe employs socket system call batching in a sim-ilar way,but it uses a standard system call interface to communicate with the kernel[28]. Batching also has been applied to packet I/O to re-duce the per-packet processing overhead.PacketShader I/O engine[27]reads and writes packets in batches and greatly improves the packet I/O performance,especially for small packets.Packet I/O batching reduces the in-terrupt,DMA,IOMMU lookup,and dynamic memory management overheads.Similar approaches are found in other high-performance packet I/O libraries[4,7,39]. In contrast,mTCP eliminates socket system calls by running the TCP stack in the user level.Also,it enforces batching from packet I/O and TCP processing up to user applications.Unlike FlexSC and MegaPipe,batching in mTCP is completely transparent without requiring kernel or user code modi?cation.Moreover,it performs batching in both directions(e.g.,packet TX and RX,application to TCP and TCP to application). Connection locality on multicore systems:TCP per-formance can be further optimized on multiprocessors by providing connection locality on the CPU cores[37].By handling all operations of same connection on the same core,it can avoid inter-core contention and unnecessary cache pollution.mTCP adopts the same idea,but applies it to both?ow-and packet-level processing. User-level TCP stacks:There have been several at-tempts to move the entire networking stack from the kernel to the user level[22,24,25,42].These are mainly(1)to ease the customizing and debugging of new network pro-tocols or(2)to accelerate the performance of existing protocols by tweaking some internal variables,such as the TCP congestion control parameters.They focus mostly on providing a?exible environment for user-level proto-col development or for exposing some in-kernel variables safely to the user level.In contrast,our focus is on build-ing a user-level TCP stack that provides high scalability on multicore systems. Light-weight networking stacks:Some applications avoid using TCP entirely for performance reasons.High performance key-value systems,such as memcached[9], Pilaf[35],and MICA[34],either use RDMA or UDP-based protocols to avoid the overhead of TCP.However, these solutions typically only apply to applications run-ning inside a datacenter.Most user-facing applications must still rely on TCP. Multikernel:Many research efforts enhance operating system scalability for multicore systems[19,20,44].Bar-rel?sh[19]and fos[44]separate the kernel resources for each core by building an independent system that manages per-core resources.For ef?cient inter-core communica-tion,they use asynchronous message passing.Corey[20] attempts to address the resource sharing problem on mul-ticore systems by having the application explicitly declare shared and local resources across multiple cores.It en-forces the default policy of having private resources for a speci?c core to minimize unnecessary contention.mTCP borrows the concept of per-core resource management from Barrel?sh,but allows ef?cient sharing between ap-plication and mTCP threads with lock-free data structures. Microkernels:The microkernel approach bears simi-larity with mTCP in that the operating system’s services run within the user level[23,30,38].Exokernel[23],for example,provides a minimal kernel and low-level inter-faces for accessing hardware while providing protection. It exposes low-level hardware access directly to the user level so that applications perform their own optimizations. This is conceptually similar to mTCP’s packet I/O library that directly accesses the NIC.mTCP,however,integrates ?ow-level and packet-level event batch processing to amor-tize the context switch overhead,which is often a critical bottleneck for microkernels. 7Conclusion mTCP is a high-performance user-level TCP stack de-signed for multicore systems.We?nd that the Linux kernel still does not ef?ciently use the CPU cycles in pro-cessing small packets despite recent improvements,and this severely limits the scalability of handling short TCP connections.mTCP unleashes the TCP stack from the ker-nel and directly delivers the bene?t of high-performance packet I/O to the transport and application layer.The key enabler is transparent and bi-directional batching of packet-and?ow-level events,which amortizes the con-text switch overhead over a batch of events.In addition, the use of lock-free data structures,cache-aware thread placement,and ef?cient per-core resource management contributes to mTCP’s performance.Finally,our evalu-ation demonstrates that porting existing applications to mTCP is trivial and mTCP improves the performance of existing applications by up to320%. Acknowledgement We would like to thank our shepherd George Porter and anonymous reviewers from NSDI2014for their valu-able comments.We also thank Sangjin Han for provid-ing the MegaPipe source code,and Sunil Pedapudi and Jaeheung Surh for proofreading the?nal version.This research is supported by the National Research Founda-tion of Korea(NRF)grant#2012R1A1A1015222and #2013R1A1A1076024. References [1]Facebook.https://https://www.wendangku.net/doc/154366542.html,/. [2]Google.https://https://www.wendangku.net/doc/154366542.html,/. [3]How long does it take to make a context switch? https://www.wendangku.net/doc/154366542.html,/2010/11/how- long-does-it-take-to-make-context.html. [4]Intel DPDK:Data Plane Development Kit.http: //https://www.wendangku.net/doc/154366542.html,/. [5]Intel VMDq Technology.https://www.wendangku.net/doc/154366542.html,/ content/dam/www/public/us/en/documents/ white-papers/vmdq-technology-paper.pdf. [6]Libevent.https://www.wendangku.net/doc/154366542.html,/. [7]Libzero for DNA:Zero-copy?exible packet pro- cessing on top of DNA.https://www.wendangku.net/doc/154366542.html,/ products/pf_ring/libzero-for-dna/. [8]Lighttpd.https://www.wendangku.net/doc/154366542.html,/. [9]memcached-a distributed memory object caching system.https://www.wendangku.net/doc/154366542.html,. [10]The NewReno modi?cation to TCP’s fast recovery algorithm.https://www.wendangku.net/doc/154366542.html,/rfc/rfc2582. txt. [11]The open group base speci?cations issue6,IEEE Std1003.1.https://www.wendangku.net/doc/154366542.html,/ onlinepubs/007904975/basedefs/pthread.h. html. [12]Packet I/O Engine.https://www.wendangku.net/doc/154366542.html,/ packetshader/io_engine/. [13]PCI-SIG SR-IOV Primer:An Introduction to SR-IOV Technology.https://www.wendangku.net/doc/154366542.html,/ content/dam/doc/application-note/pci- sig-sr-iov-primer-sr-iov-technology- paper.pdf. [14]The SO REUSEPORT socket option.https://lwn. net/Articles/542629/. [15]The Apache HTTP Server Project.http://httpd. https://www.wendangku.net/doc/154366542.html,/. [16]The Apache Portable Runtime Project.http://apr. https://www.wendangku.net/doc/154366542.html,/. [17]Transmission control protocol.http://www.ietf. org/rfc/rfc793.txt. [18]Twitter.https://https://www.wendangku.net/doc/154366542.html,/.[19]A.Baumann,P.Barham,P.-E.Dagand,T.Harris, R.Isaacs,S.Peter,T.Roscoe,A.Sch¨u pbach,and A.Singhania.The multikernel:a new OS architec- ture for scalable multicore systems.In Proceedings of the ACM SIGOPS symposium on Operating sys-tems principles(SOSP),2009. [20]S.Boyd-Wickizer,H.Chen,R.Chen,Y.Mao, F.Kaashoek,R.Morris,A.Pesterev,L.Stein,M.Wu, Y.Dai,Y.Zhang,and Z.Zhang.Corey:an operat-ing system for many cores.In Proceedings of the USENIX conference on Operating systems design and implementation(OSDI),2008. [21]M.Dobrescu,N.Egi,K.Argyraki,B.-G.Chun, K.Fall,G.Iannaccone,A.Knies,M.Manesh,and S.Ratnasamy.RouteBricks:exploiting parallelism to scale software routers.In Proceedings of the ACM SIGOPS Symposium on Operating Systems Princi-ples(SOSP),2009. [22]D.Ely,S.Savage,and D.Wetherall.Alpine:a user- level infrastructure for network protocol develop-ment.In Proceedings of the conference on USENIX Symposium on Internet Technologies and Systems (USIT),2001. [23]D.R.Engler,M.F.Kaashoek,and J.O’Toole,Jr. Exokernel:an operating system architecture for application-level resource management.In Proceed-ings of the ACM symposium on Operating systems principles(SOSP),1995. [24]G.R.Ganger,D.R.Engler,M.F.Kaashoek,H.M. Brice?n o,R.Hunt,and T.Pinckney.Fast and?exible application-level networking on exokernel systems. ACM Transactions on Computer Systems(TOCS), 20(1):49–83,Feb.2002. [25]H.S.Gunawi,A.C.Arpaci-Dusseau,and R.H. Arpaci-Dusseau.Deploying safe user-level network services with ictcp.In Proceedings of the confer-ence on Symposium on Opearting Systems Design& Implementation(OSDI),2004. [26]S.Ha,I.Rhee,and L.Xu.CUBIC:a new TCP- friendly high-speed TCP variant.SIGOPS Oper.Syst. Rev.,42(5):64–74,July2008. [27]S.Han,K.Jang,K.Park,and S.B.Moon.Pack- etShader:a GPU-accelerated software router.In Proceedings of the ACM Special Interest Group on Data Communication(SIGCOMM),2010. [28]S.Han,S.Marshall,B.-G.Chun,and S.Ratnasamy. MegaPipe:a new programming interface for scal-able network I/O.In Proceedings of the USENIX conference on Operating Systems Design and Imple-mentation(OSDI),2012. [29]H.H¨a rtig,M.Hohmuth,J.Liedtke,S.Sch¨o nberg, and J.Wolter.The performance ofμ-kernel-based systems.In Proceedings of the ACM Symposium on Operating Systems Principles(SOSP),1997. [30]H.H¨a rtig,M.Hohmuth,J.Liedtke,J.Wolter,and S.Sch¨o nberg.The performance ofμ-kernel-based systems.SIGOPS Oper.Syst.Rev.,31(5):66–77,Oct. 1997. [31]S.Ihm and V.S.Pai.Towards understanding mod- ern web traf?c.In Proceedings of the ACM SIG-COMM conference on Internet measurement confer-ence(IMC),2011. [32]K.Jang,S.Han,S.Han,S.Moon,and K.Park. SSLShader:cheap SSL acceleration with commod-ity processors.In Proceedings of the USENIX con-ference on Networked systems design and implemen-tation(NSDI),2011. [33]R.Kapoor,G.Porter,M.Tewari,G.M.V oelker, and A.Vahdat.Chronos:Predictable Low Latency for Data Center Applications.In Proceedings of the Third ACM Symposium on Cloud Computing (SOCC),2012. [34]H.Lim,D.Han,D.G.Andersen,and M.Kamin- sky.MICA:A Holistic Approach to Fast In-Memory Key-Value Storage.In Proceedings of the USENIX Symposium on Networked Systems Design and Im-plementation(NSDI),2014. [35]C.Mitchell,Y.Geng,and https://www.wendangku.net/doc/154366542.html,ing one-sided RDMA reads to build a fast,CPU-ef?cient key-value store.In Proceedings of the USENIX Annual Techni-cal Conference(ATC),2013. [36]R.Nishtala,H.Fugal,S.Grimm,M.Kwiatkowski, H.Lee,H.C.Li,R.McElroy,M.Paleczny,D.Peek, P.Saab,et al.Scaling memcache at Facebook.In Pro-ceedings of the USENIX conference on Networked Systems Design and Implementation(NSDI),2013. [37]A.Pesterev,J.Strauss,N.Zeldovich,and R.T.Mor- ris.Improving network connection locality on multi-core systems.In Proceedings of the ACM european conference on Computer Systems(EuroSys),2012. [38]R.Rashid,D.Julin,D.Orr,R.Sanzi,R.Baron, A.Forin,D.Golub,and M.Jones.Mach:A system software kernel.In Proceedings of the Computer So-ciety International Conference(COMPCON),1989.[39]https://www.wendangku.net/doc/154366542.html,map:a novel framework for fast packet I/O.In Proceedings of the USENIX conference on Annual Technical Conference(ATC),2012. [40]L.Soares and M.Stumm.FlexSC:?exible system call scheduling with exception-less system calls.In Proceedings of the USENIX conference on Operating systems design and implementation(OSDI),2010. [41]L.Soares and M.Stumm.Exception-less system calls for event-driven servers.In Proceedings of the USENIX conference on USENIX annual technical conference(ATC),2011. [42]C.A.Thekkath,T.D.Nguyen,E.Moy,and E.D. Lazowska.Implementing network protocols at user level.IEEE/ACM Transactions on Networking (TON),1(5):554–565,Oct.1993. [43]V.Vasudevan,D.G.Andersen,and M.Kaminsky. The case for VOS:the vector operating system.In Proceedings of the USENIX conference on Hot topics in operating systems(HotOS),2011. [44]D.Wentzlaff and A.Agarwal.Factored operating systems(fos):the case for a scalable operating sys-tem for multicores.ACM SIGOPS Operating Sys-tems Review,43(2):76–85,Apr.2009. [45]S.Woo,E.Jeong,S.Park,J.Lee,S.Ihm,and K.Park. Comparison of caching strategies in modern cellu-lar backhaul networks.In Proceeding of the annual international conference on Mobile systems,appli-cations,and services(MobiSys),2013. 讨论ZigBee协议栈的构成以及内部OSAL的工作机理。 ZigBee协议栈OSAL介绍 操作系统抽象层 OSAL常用术语： 1.资源（Resource）：任何任务所占用的实体都叫资源，如变量、数组、结构体 2.共享资源（Shared Resource）：两个或两个以上任务使用的资源，为防止破坏资源，任务在操作共享资源时是独占状态。 3.任务（Task）：即线程，简单的程序的执行过程。任务设计时将问题尽可能分成多个任务，每个任务独立完成某项功能，同时赋予优先级、CPU寄存器和堆栈空间。一般一个任务设计为一个无限循环。 4.多任务运行（Muti-task Running）：其实同一时刻只有一个任务运行。 5.内核（Kernel）：内核负责管理各个任务。包括：分配CPU时间；任务调度；任务间的通信。 6.互斥（Mutual Exclusion）：多任务通信最常用方法是共享数据结构。 保护共享资源常用的方法： 关中断； 使用测试并置位指令（T&S指令）； 禁止任务切换； 使用信号量； 7.消息队列（Message Queue）：用于任务间传递消息。 OSAL提供如下功能： 任务注册、初始化和启动； 任务间的同步、互斥； 中断处理； 储存器分配和管理； OSAL运行机理： OSAL就是一种支持多任务运行的系统资源分配机制。 OSAL是一种基于事件驱动的轮询式操作系统。、 void osal_start_system(void)是ZigBee协议栈的灵魂，不断的查看事件列表，如果有事件发生就调用相应的事件处理函数。 SYS_EVENT_MSG是一个事件集合，是由协议栈定义的事件，即系统强制事件（Mandatory Events），它的定义为： #define SYS_EVENT_MSG 0x8000； 它包含如下事件： AF_INCOMING_MSG_CMD 收到一个新的无线数据 1.概述 1.1解析ZigBee堆栈架构 ZigBee堆栈是在IEEE 802.15.4标准基础上建立的，定义了协议的MAC和PHY层。ZigBee设备应该包括IEEE802.15.4(该标准定义了RF射频以及与相邻设备之间的通信)的PHY和MAC层，以及ZigBee堆栈层：网络层(NWK)、应用层和安全服务提供层。图1-1给出了这些组件的概况。 1.1.1ZigBee堆栈层 每个ZigBee设备都与一个特定模板有关，可能是公共模板或私有模板。这些模板定义了设备的应用环境、设备类型以及用于设备间通信的簇。公共模板可以确保不同供应商的设备在相同应用领域中的互操作性。 设备是由模板定义的，并以应用对象(Application Objects)的形式实现(见图1-1)。每个应用对象通过一个端点连接到ZigBee堆栈的余下部分，它们都是器件中可寻址的组件。 图1-1 zigbe堆栈框架 从应用角度看，通信的本质就是端点到端点的连接(例如，一个带开关组件的设备与带一个或多个灯组件的远端设备进行通信，目的是将这些灯点亮)。 端点之间的通信是通过称之为簇的数据结构实现的。这些簇是应用对象之间共享信息所需的全部属性的容器，在特殊应用中使用的簇在模板中有定义。图1-1-2就是设备及其接口的一个例子： 图1-1-2 每个接口都能接收(用于输入)或发送(用于输出)簇格式的数据。一共有二个特殊的端点，即端点0和端点255。端点0用于整个ZigBee设备的配置和管理。应用程序可以通过端点0与ZigBee 堆栈的其它层通信，从而实现对这些层的初始化和配置。附属在端点0的对象被称为ZigBee设备对象 (ZD0)。端点255用于向所有端点的广播。端点241到254是保留端点。 所有端点都使用应用支持子层(APS)提供的服务。APS通过网络层和安全服务提供层与端点相接，并为数据传送、安全和绑定提供服务，因此能够适配不同但兼容的设备，比如带灯的开关。APS使用网络层(NWK)提供的服务。NWK负责设备到设备的通信，并负责网络中设备初始化所包含的活动、消息路由和网络发现。应用层可以通过ZigBee设备对象(ZD0)对网络层参数进行配置和访问。 1.1.2 80 2.15.4 MAC层 IEEE 802.15.4标准为低速率无线个人域网(LR-WPAN)定义了OSI模型开始的两层。PHY层定义了无线射频应该具备的特征，它支持二种不同的射频信号，分别位于2450MHz波段和868/915MHz 波段。2450MHz波段射频可以提供250kbps的数据速率和16个不同的信道。868 /915MHz波段中，868MHz支持1个数据速率为20kbps的信道，915MHz支持10个数据速率为40kbps的信道。MAC层负责相邻设备间的单跳数据通信。它负责建立与网络的同步，支持关联和去关联以及MAC 层安全：它能提供二个设备之间的可靠链接。 1.1.3 关于服务接入点 ZigBee堆栈的不同层与802.15.4 MAC通过服务接入点(SAP)进行通信。SAP是某一特定层提供的服务与上层之间的接口。 ZigBee堆栈的大多数层有两个接口：数据实体接口和管理实体接口。数据实体接口的目标是向上层提供所需的常规数据服务。管理实体接口的目标是向上层提供访问内部层参数、配置和管理数据的机制。 1.1.4 ZigBee的安全性 安全机制由安全服务提供层提供。然而值得注意的是，系统的整体安全性是在模板级定义的，这意味着模板应该定义某一特定网络中应该实现何种类型的安全。 每一层(MAC、网络或应用层)都能被保护，为了降低存储要求，它们可以分享安全钥匙。SSP是通过ZD0进行初始化和配置的，要求实现高级加密标准(AES)。ZigBee规范定义了信任中心的用 SS7信令系统协议简介 SS7信令协议栈，MTP1，MTP2，MTP3，SCCP，TCAP，ISUP，TUP 3.1 SS7信令协议栈 协议是通过网络传送数据的规则集合。 协议栈也就是协议的分层结构，协议分层的目的是为了使各层相对独立，或使各层具有不同的职能。SS7协议一开始就是按分层结构的思想设计的，但SS7协议 在开始发展时，主要是考虑在数字电话网和采用电路交换方式的数据通信网中传送各种与电路有关的信息，所以CCITT在80年代提出的SS7技术规范黄皮书 中对SS7协议的分层方法没有和OSI七层模型取得一致，对SS7协议只提出了4个功能层的要求。这4个功能层如下： 物理层：就是底层，具体是DS0或V.35。 数据链路层：在两节点间提供可靠的通信。 网络层：提供消息发送的路由选择.。 用户部份／应用部份：就是数据库事务处理，呼叫建立和释放。 但随着综合业务数字网（ISDN）和智能网的发展，不仅需要传送与电路有关的消息，而且需要传送与电路无关的端到端的消息，原来的四层结构已不 能满足要求。在1984年和1988年的红皮书和蓝皮书建议中，CCITT作了大量的努力，使SS7协议的分层结构尽量向OSI的七层模型靠近。 下图图示了SS7信令协议栈： MTP1(消息传递部分第一层)：即物理层。 MTP1(消息传递部分第二层)：即数据链路层。 MTP1(消息传递部分第三层)：即网络层。 SCCP（信令连接控制部分） TCAP（事务处理应用部分） ISUP（ISDN用户部分） TUP（电话用户部分） MTP1 MTP1是SS7协议栈中的最底层，对应于OSI模型中的物理层，这一层定义了数字链路在物理上，电气上及功能上的特性。物理接口的定义包括：E－1，T－1，DS －1，V.35,DS－0，DS －0A（56K）。 MTP2 MTP2确保消息在链路上实现精确的端到端传送。MTP2提供流控制，消息序号，差错检查等功能。当传送出错时，出错的消息会被重发。MTP2对应OSI模型中的数据链路层。 MTP3 MTP3在SS7信令网中提供两个信令点间消息的路由选择功能，消息在依次通过MTP1，MTP2，MTP3层之后，可能会 被发送回MTP2再传向别的信令点，也可能会传递给某个应用层，如：SCCP或ISUP 层。MTP3还提供一些网管功能的支持，包括：流量控制，路由选择 和链路管理。MTP3对应OSI模型中的网络层。 SCCP（信令连接控制部分） SCCP位于MTP之上，为MTP提供附加功能，以便通过SS7信令网在信令点之间传递电路相关和非电 路相关的消息，提供两类无连接业务和两类面向连接的业务。 无连接业务是指在两个应用实体间，不需要建立逻辑连接就可以传递信令数据。面向连接的业务在数据传递之前应用实体之间必须先建立连接，可以是一般性的连 1Zigbee协议栈相关概念 1.1近距离通信技术比较： 近距离无线通信技术有wifi、蓝牙、红外、zigbee，在无线传感网络中需求的网络通信恰是近距离需求的，故，四者均可用做无线传感网络的通信技术。而，其中（1）红外（infrared）：能够包含的信息过少；频率低波衍射性不好只能视距通信；要求位置固定；点对点传输无法组网。（2）蓝牙（bluetooth）：可移动，手机支持；通信距离10m；芯片价格贵；高功耗（3）wifi：高带宽；覆盖半径100m；高功耗；不能自组网；（4）zigbee:价格便宜；低功耗；自组网规模大。?????WSN中zigbee通信技术是最佳方案，但它连接公网需要有专门的网关转换→进一步学习stm32。 1.2协议栈 协议栈是网络中各层协议的总和，其形象的反映了一个网络中文件传输的过程：由上层协议到底层协议，再由底层协议到上层协议。 1.2.1Zigbee协议规范与zigbee协议栈 Zigbee各层协议中物理层（phy）、介质控制层（mac）规范由IEEE802.15.4规定，网络层（NWK）、应用层（apl）规范由zigbee联盟推出。Zigbee联盟推出的整套zigbee规范：2005年第一版ZigBeeSpecificationV1.0，zigbee2006，zigbee2007、zigbeepro zigbee协议栈：很多公司都有自主研发的协议栈，如TI公司的：RemoTI，Z-Stack，SimpliciTI、freakz、msstatePAN 等。 1.2.2z-stack协议栈与zigbee协议栈 z-stack协议栈与zigbee协议栈的关系：z-stack是zigbee协议栈的一种具体实现，或者说是TI公司读懂了zigbee 协议栈，自己用C语言编写了一个软件—---z-stack，是由全球几千名工程师共同开发的。ZStack-CC2530-2.3.1-1.4.0软件可与TI的SmartRF05平台协同工作，该平台包括MSP430超低功耗微控制器(MCU)、CC2520RF收发器以及CC2591距离扩展器，通信连接距离可达数公里。 Z-Stack中的很多关键的代码是以库文件的形式给出来，也就是我们只能用它们，而看不到它们的具体的实现。其中核心部分的代码都是编译好的，以库文件的形式给出的，比如安全模块，路由模块，和Mesh自组网模块。与z-stack 相比msstatePAN、freakz协议栈都是全部真正的开源的，它们的所有源代码我们都可以看到。但是由于它们没有大的商业公司的支持，开发升级方面，性能方面和z-stack相比差距很大，并没有实现商业应用，只是作为学术研究而已。 还可以配备TI的一个标准兼容或专有的网络协议栈（RemoTI，Z-Stack，或SimpliciTI）来简化开发,当网络节点要求不多在30个以内，通信距离500m-1000m时用simpliciti。 1.2.3IEEE802.15.4标准概述 IEEE802.15.4是一个低速率无线个人局域网(LowRateWirelessPersonalAreaNetworks，LR-WPAN)标准。定义了物理层(PHY)和介质访问控制层(MAC)。 LR-WPAN网络具有如下特点： ◆实现250kb/s，40kb/s，20kb/s三种传输速率。 ◆支持星型或者点对点两种网络拓扑结构。 ◆具有16位短地址或者64位扩展地址。 ◆支持冲突避免载波多路侦听技术(carriersensemultipleaccesswithcollisionavoidance，CSMA/CA)。（mac层） ◆用于可靠传输的全应答协议。(RTS-CTS) ◆低功耗。 ◆能量检测(EnergyDetection，ED)。 ◆链路质量指示(LinkQualityIndication，LQI)。 ◆在2.45GHz频带内定义了16个通道；在915MHz频带内定义了10个通道；在868MHz频带内定义了1个通道。 为了使供应商能够提供最低可能功耗的设备，IEEE(InstituteofElectricalandElectronicsEngineers，电气及电子工程师学会)定义了两种不同类型的设备：一种是完整功能设备(full．functionaldevice，FFD)，另一种是简化功能设备 竭诚为您提供优质文档/双击可除 arp协议栈 篇一：实验2地址解析协议aRp 实验2地址解析协议（aRp） 【实验目的】 1.掌握aRp协议的报文格式 2.掌握aRp协议的工作原理 3.理解aRp高速缓存的作用 4.掌握aRp请求和应答的实现方法 5.掌握aRp缓存表的维护过程 【学时分配】 2学时 【实验环境】 该实验采用网络结构二 【实验原理】 一、物理地址与逻辑地址 1.物理地址 物理地址是节点的地址，由它所在的局域网或广域网定义。物理地址包含在数据链路层的帧中。物理地址是最低一 级的地址。 物理地址的长度和格式是可变的，取决于具体的网络。以太网使用写在网络接口卡（nic）上的6字节的标识作为 物理地址。 物理地址可以是单播地址（一个接收者）、多播地址（一组接收者）或广播地址（由网络中的所有主机接收）。有些 网络不支持多播或广播地址，当需要把帧发送给一组主机或所有主机时，多播地址或广播地址就需要用单播地址来模拟。 2.逻辑地址 在互联网的环境中仅使用物理地址是不合适的，因为不同网络可以使用不同的地址格式。因此，需要一种通用的编址系统，用来惟一地标识每一台主机，而不管底层使用什么样的物理网络。 逻辑地址就是为此目的而设计的。目前internet上的 逻辑地址是32位地址，通常称为ip地址，可以用来标识连接在internet上的每一台主机。在internet上没有两个主机具有同样的ip地址。 逻辑地址可以是单播地址、多播地址和广播地址。其中广播地址有一些局限性。在实验三中将详细介绍这三种类型的地址。 二、aRp协议简介 internet是由各种各样的物理网络通过使用诸如路由 蓝牙协议栈BlueZ的移植与开发* 欧阳鑫 于红岩 吕杨 (昆明理工大学信息工程与自动化学院，昆明，650051) 摘要：蓝牙技术是当前国内外科技界和产业界研究开发的热点技术，其应用范围包括手机、PDA、信息家电设备等领域，蓝牙技术在嵌入式系统上必将得到广泛的应用。而要在嵌入式系统上提供蓝牙开发支持，蓝牙协议栈的移植是关键。本文分析了蓝牙协议栈BlueZ体系结构，详细介绍了在S3C2410开发板上移植BlueZ的步骤，建立了嵌入式蓝牙应用开发平台，并提出了用BlueZ 开发蓝牙应用程序的思路。 关键字：蓝牙技术；蓝牙协议栈BlueZ；移植；S3C2410；Linux 中图分类号：TP368.1 文献标识码：A BlueZ Porting and Programming Ou Yangxin，Yu Hongyan，Lv Yang (College of Information Engineering and Automation，Kunming University of Science and Technology，Kunming 650051，China) Abstract: Bluetooth technology is the focused on by the domestic and overseas research institutes. This technology is widely used in mobile phone, PDA, and Information Appliance devices. In the future，bluetooth technology will be widely used in embedded system. Built some bluetooth applications on embedded system，the key technology is porting bluetooth protocol suites to platform. In this paper，we study the BlueZ architecture，describe the steps of Porting BlueZ to S3C2410 Platform in detail，and build the embedded bluetooth application programming platform. In the end, we also give the idea of bluetooth programming. Keywords: bluetooth technology，bluetooth protocol suites BlueZ，porting, S3C2410, Linux 1引言 蓝牙技术是一项低价格、低功耗的射频技术，它能使蓝牙设备实现近距离无线通信。由于蓝牙技术有广泛的应用前景，它已成为当前国内外科技界和产业界研究开发的热点技术。Linux 操作系统的开放的蓝牙协议栈主要包括IBM公司的BlueDrekar，Nokia公司的Affix, Axis公司的OpenBT和官方协议栈BlueZ[1]。BlueZ是公布在Internet上的免费蓝牙协议栈，由于它结构简单，应用方便，具有灵活、高效和模块化的特点且具有较强的兼容性，因此BlueZ已经成为Linux操作系统下的官方的蓝牙协议栈。 S3C2410x是三星公司推出的一款高性价比32位的RISC处理器，内含一个由ARM公司设计的ARM920T核，具有低功耗高性能的特点，适用于对价格及功耗敏感的场合。本文使用的S3C2410开发板主要包含以下部件：S3C2410x芯片，32MB Nor Flash，64MB SDRAM，IIC存储器接口，LCD控制器，UART接口，一个USB(Host)接口。 利用S3C2410开发板上的USB接口，可以外扩蓝牙适配器，但S3C2410开发板上没有实现蓝牙设备驱动。本文对蓝牙协议栈BlueZ进行分析，移植BlueZ到开发板上实现蓝牙设备驱动，提出了使用BlueZ开发蓝牙应用程序的基本思路。 *基金项目：云南省自然科学基金项目(2004F0024M)。 DVB / ETSI EN-50221 Common Interface Stack The GkWare Common Interface Stack already enables PayTV on thousands of DVB Setttop boxes worldwide, including single-slot and dual-slot systems and little and big-endian CPUs. The ANSI-C sourcecode is portable and only a small lowlevel PCMCIA I/O driver has to be developed for new platform integrations. Compatibility for all modules available to the general public is guaranteed. FEATURES Support for SCM CiMax, I&C StarCI and direct GPIO connections Simple Integration of custom Resources Flexible PMT => CAPMT converter PCMCIA Card-Info-Structure decoder included Full ETSI R206-001 profile level 1 implementation PVR / Headend descrambling mode, including support for Aston Professional CAM series Supports friendly coexistence with other PCMCIA drivers (e.g. Compact Flash, Network Interface or Bluetooth) SUPPORTED EN50221 RESOURCES Resource Manager Application Information Version 1 and 2 MMI DateTime CA Support Host Control LowSpeed Communication SUPPORTED PLATFORMS ARM7 (Thumb) ARM9 MIPS ST (OS20) x86 REQUIREMENTS approximately 32kb RAM + 16kb per Slot Multitasking OS recommended (but not required) (existing ports for Nucleus+, uCOS, pSoS, Win32) SUPPORTED CHIPSETS Conexant CX2249x, CX241xx, CX2417x ST 5105, 5519 Philips LPC 2214 NEC EMMA Series TI AV711x Technotrend DVB-PCI Budget series (including clones) KNC-One TV-Station Cineview slot (including clones) Whatever CI module you have... we make it work ! MMI Menu Sample Various licensing models are available, including low-volume closed-source and royalty-free with a limited sourcecode redistribution license. GkWare e.K. - Humboldtstrasse 177 - 45149 Essen - Germany https://www.wendangku.net/doc/154366542.html, - support@https://www.wendangku.net/doc/154366542.html, - +49 174 5208026 TI Z-stack协议栈开发环境和工作流程 系统软件设计是在硬件设计的基础上进行的，良好的软件设计是实现系统功能的重要环节，也是提高系统性能的关键所在。节点设计基于通用性及便于开发的考虑，移植了TI公司的Z-Stack协议栈，其主要特点就是其兼容性，完全支持IEEE 802.15.4/ZigBee的CC2430片上系统解决方案。Z-Stack还支持丰富的新特性，如无线下载，可通过ZigBee网状网络（Mesh Network）下载节点更新。 图 ZigBee节点开发环境 TI的Z-Stack装载在一个基于IAR开发环境的工程里。强大的IAR Embedded Workbench除了提供编译下载功能外，还可以结合编程器进行单步跟踪调试和监测片上寄存器、Flash数据等。Z-Stack根据IEEE 802.15.4和ZigBee标准分为以下几层：API（Application Programming Interface），HAL（Hardware Abstract Layer），MAC（Media Access Control)，NWK（Zigbee Network Layer），OSAL （Operating System Abstract System），Security，Service，ZDO（Zigbee Device Objects）。使用IAR打开工程文件SampleApp.eww后，即可查看到整个协议栈从HAL层到APP层的文件夹分布。该协议栈可以实现复杂的网络链接，在协调器节点中实现对路由表和绑定表的非易失性存储，因此网络具有一定的记忆功能。 Z-Stack采用操作系统的思想来构建，采用事件轮循机制，当各层初始化之后，系统进入低功耗模式，当事件发生时，唤醒系统，开始进入中断处理事件，结束后继续进入低功耗模式。如果同时有几个事件发生，判断优先级，逐次处理事件。这种软件构架可以极大地降级系统的功耗。 本系列教程将结合TI推出的CC254x SoC 系列，讲解从环境的搭建到蓝牙4.0协议栈的开发来深入学习蓝牙4.0的开发过程。教程共分为六部分，本文为第五部分： 第五部分知识点： 第二十一节 DHT11温湿度传感器 第二十二节蓝牙协议栈之从机通讯 第二十三节蓝牙协议栈主从一体之主机通讯 第二十四节 OAD空中升级 第二十五节 SBL串口升级 有关TI 的CC254x芯片介绍，可点击下面链接查看： 主流蓝牙BLE控制芯片详解（1）：TI CC2540 同系列资料推荐： 由浅入深，蓝牙4.0/BLE协议栈开发攻略大全（1） 由浅入深，蓝牙4.0/BLE协议栈开发攻略大全（2） 由浅入深，蓝牙4.0/BLE协议栈开发攻略大全（3） 由浅入深，蓝牙4.0/BLE协议栈开发攻略大全（4） 有关本文的工具下载，大家可以到以下这个地址： 朱兆祺ForARM 第二十一节 DHT11温湿度传感器 DHT11简介 DHT11数字温湿度传感器是一款含有已校准数字信号输出的温湿度复合传感器，它应用专用的数字模块采集技术和温湿度传感技术，确保产品具有极高的可靠性和卓越的长期稳定性。传感器包括一个电阻式感湿元件和一个NTC测温元件，并与一个高性能8位单片机相 连接。因此该产品具有品质卓越、超快响应、抗干扰能力强、性价比极高等优点。每个DHT11传感器都在极为精确的湿度校验室中进行校准。校准系数以程序的形式存在OTP内存中，传感器内部在检测型号的处理过程中要调用这些校准系数。单线制串行接口，使系统集成变得简易快捷。超小的体积、极低的功耗，使其成为给类应用甚至最为苛刻的应用场合的最佳选择。产品为4针单排引脚封装，连接方便。 技术参数 供电电压： 3.3~5.5V DC 输出：单总线数字信号 测量范围：湿度20-90%RH，温度0~50℃ 测量精度：湿度+-5%RH，温度+-2℃ 分辨率：湿度1%RH，温度1℃ 互换性：可完全互换， 长期稳定性： < ±1%RH/年 DHT11 数字湿温度传感器采用单总线数据格式。即，单个数据引脚端口完成输入输出双向传输。其数据包由 5Byte（40Bit）组成。数据分小数部分和整数部分，一次完整的数据传输为40bit，高位先出。DHT11 的数据格式为：8bit 湿度整数数据+8bit 湿度小数数据+8bit 温度整数数据+8bit 温度小数数据+8bit 校验和。其中校验和数据为前四个字节相加。传感器数据输出的是未编码的二进制数据。数据（湿度、温度、整数、小数）之间应该分开处理。例如，某次从 DHT11 读到的数据如图所示： 协议栈DHT11测试 协议栈工作原理介绍 CC2540集成了增强型的8051内核，TI为BLE协议栈搭建了一个简单的操作系统，即一种任务轮询机制。帮你做好了底层和蓝牙协议深层的内容，将复杂部分屏蔽掉。让用户通过API函数就可以轻易用蓝牙4.0，是开发起来更加方便，开发周期也可以相应缩短。 1.1.1工程文件介绍 安装完BLE协议栈之后，会在安装目录下看到以下文件结构： 图 3.2BLE栈目录 可看到Projects文件夹里面有很多工程，我们主要介绍SimpleBLECentral和SimpleBLEPeripheral。 ble文件夹中有很多工程文件，有些是具体的应用，例如 BloodPressure、GlucoseCollector、GlucoseSensor、HeartRate、HIDEmuKbd等都为传感器的实际应用，有相应标准的Profile（即通用的协议）。 其中还有4中角色：SimpleBLEBroadcaster、SimpleBLECentral、SimpleBLEObserver、SimpleBLEPeripheral。 他们都有自己的特点。 ?Broadcaster广播员——非连接性的信号装置 ?Observer观察者——扫描得到，但不能链接 ?Peripheral从机——可链接，在单个链路层链接中作为从机?Central主机——扫描设备并发起链接，在单链路层或多链路层 中作为主机。 最后的BTool文件夹为BLE设备PC端的使用工具。 1.1.2OSAL介绍 协议栈是一个小操作系统。大家不要听到是操作系统就感觉到很复杂。回想 我们当初学习51单片机时候是不是会用到定时器的功能？嗯，我们会利用定时器计时，令LED一秒改变一次状态。好，现在进一步，我们利用同一个定时器计时，令LED1一秒闪烁一次，LED2二秒闪烁一次。这样就有2个任务了。再进一步…有n个LED,就有n个任务执行了。协议栈的最终工作原理也一样。从它工作开始，定时器周而复始地计时，有发送、接收…等任务要执行时就执行。这个方式称为任务轮询。 图 3.3任务轮询 现在我们直接打开协议栈，直接拿他们的东西来解剖！我们打开协议栈文件夹Texas Instruments\BLE-CC254x-1.2.1\Projects \ble\SimpleBLEPeripheral\CC2540DB里面的工程文件SampleApp.eww。 协议栈开发总结范文 xx年5月30日星期六晴近来一直很少些关于技术方面的文章，一来是被工作和生活所累，没有很多闲暇多余的时间,二来呢是觉得一直没有好的case，或则有好的case，但觉得目前在这个case方面，自己还不算很professional。 今天心情不错，感觉也很好，就把我近一年来在“协议栈”开发方面的一些经验写下来，算是对自己这一阶段的一个终结，也希望能对后来者能有所帮助。 首先，不要以为做协议栈开发很难(诚然，商用稳定的协议栈开发的确也不容易^_^)，所以首先一定要有足够的信心，其次如果有前辈带你上手那就好很多，如果没有，自己一个人去“专研”，那么你做的首要的事情还是要有信心，不要有“惧怕”的思绪。 好了，开始进入正题，如下一，阅读协议栈相关文档关于单个协议的定义，权威的自然是IETF（互联网工程任务推进组织）的"rfc"文档，虽然是一大堆的英文（呵呵，我英文也一般），开始不太看的明白，不过没关系，先去网络上用"baidu"或则"google"搜索一下关于你要做的这个协议的中文说明，虽然绝大数介绍都很肤浅，但这么做的在于去了解该协议的目的和用途，心中有个大的概念，而后在大致的阅读下该协议"rfc"文档的大体描述和结构。 需要说明的是在大多数情况下，一个完整的协议栈都有好几个协议组成，所以也有1个或若干个"rfc"文档要去学习和了解，但切记 不用把所有"rfc"文档都读的相当仔细和完全明白后再去做，我们的目的在于只把握协议栈的框架和大概信息。 所以，这个阶段是基础准备阶段。 二，设计协议栈结构这个阶段也是最核心、最重要的一个阶段，可以说，开发成功与否的关键也全在这个阶段。 我们在第一个阶段的基础上，明白了要开发的协议栈的功能及其相关协议后，我们就要来设计这个协议栈框架的实现，一般有注意如下几方面1，功能性这个很自然，如能实现功能还做什么2，可扩展协议栈最好划分出内核和外围模块，利于将来扩展和维护3，模块独立协议栈内核和外围模块尽量独立，减少耦合这个阶段虽然说起来容易，但做起来去要看个人的能力和水平，包括代码的机构、风格、易维护、易移植、稳定、健壮性等等。 所以，如果连基本程序都写不好，没有好的代码风格和没有把握大结构能力的，最好还是请别人帮忙。 注释协议栈的设计要看协议栈的特点，比如有的协议栈是对称的（比如rtsp流控协议），有些是不对称的(比如协议)，各有各特点，所以在把握大结构的同时要针对各自的特点来设计。 三，编码开发框架完成后，开始丰满核心模块和构建基本的外围模块。 通常这阶段需要搭建好开发环境，便测试便修改。 注释很重要的工作，ethereal(抓包软件)，做协议这个是一定要用熟的。 协议栈的层次 SIP为应用层(Application-Layer)的协议，所以不需要改变操作系统便可以支持。SIP 已经获得3GPP (Third GenerationPartnership Project)、3GPP2 (Third Generation Partnership ProjectNumber 2)等机构认证，成为未来第三代行动通讯 (3G) 的标准。 下面是SIP的分层图示，IETF坚持分层，不同模块功能相对独立，各层之间松散耦合。 关于Resiprocate设计 首先祭出这面大旗,”类是对概念的描述，面向接口编程;封装变化的概念。”---这不是我讲的,是大师们的口水。 Resiprocate中大部分类就是对RFC3261各种SIP元素、组件的封装，并且也体现了RFC协议设计的层次。 在面向对象的设计中我们首先就要厘清问题域的所在；SIP Stack的设计就是要充分考虑完整展现RFC定义的各种元素和概念以及让这些独立而又关联的元素互动起来成为一个活的系统。 可以这样来考虑,比如我们知道RFC定义了一个SIP MESSAGE的概念；下面是从RFC文档拷贝的内容： SIP 消息 = 起始行 *消息头部 CRLF(空行) [消息体] 因此SIP Message这个概念元素还包括了更多的元素和概念；SIP Message中我们能抽象出更通用的概念我们暂且叫它Message; 起始行的概念E文RequestLine以及Sattus Line又包括了很多消息头（这是包容的关系），SIPURL也包括消息头，等等，还有什么参数什么的元素呢；当我们在考虑和提炼这些概念和元素的时候，我们思考怎么抽象他们呢，它们又有什么基本的元素及其共性呢？他们之间的关系如何组织呢？esiprocate的源码告诉了我们如何去设计和封装这些概念的上佳实现。在Resiprocate 中一些RFC3261中定义元素的对应： 建议：利用CRC卡片的方式去记录理解Resiprocate中的大量的类及其关系。CRC：类、职责、协作。 部分设计的理解： OBSERVER/VISITOR/COMMAND/ITERATOR模式，工厂模式(大量容器的使用也是一种变体如:DialogSet)，代理类句柄类（界面实现分离，隐藏实现…）,…… OMCI协议(1) 1、OMCI协议栈的结构 GPON 系统的协议栈，主要由物理媒质相关（PMD）层和GPON 传输汇聚（GTC）层组成。GTC 层从结构层次来分可以分成两个子层：GTC 成帧子层和TC 适配子层。从功能层次可以分为C/M平面和U平面。GTC 层可分为两种封装模式：ATM 模式和GEM 模式，目前GPON 设备基本都采用GEM 模式。GEM 模式的GTC 层可为其客户层提供3 种类型的接口：ATM 客户接口、GEM客户接口和ONT 管理和控制接口（OMCI） 2、PMD层 GPON 的PMD 层对应于OLT 和ONU 之间的光传输接口（也称为PON 接口），其具体参数值决定了GPON 系统的最大传输距离和最大分路比。OLT 和ONU 的发送光功率、接收机灵敏度等关键参数主要根据系统支持的ODN 类型来进行划分。根据允许衰减范围的不同，ODN 类型主要分为A、B、C 三大类，结合目前实际应用需求和光收发模块的实际能力工业界还定义了B+类，扩展了GPON 系统支持 1244.16 Mbit/s/155.52 Mbit/s； 1244.16 Mbit/s/622.08 Mbit/s； 1244.16 Mbit/s/1244.16 Mbit/s； 2488.32 Mbit/s/155.52 Mbit/s； 2488.32 Mbit/s/622.08 Mbit/s； 2488.32 Mbit/s/1244.16 Mbit/s； 2488.32 Mbit/s/2488.32 Mbit/s。 目前主流厂家的GPON 产品均支持2488.32Mbit/s/1244.16Mbit/s，并且在20km 传输距离下支持1:64 分路比。 OMCI协议（2） 1、GTC层 TC 层（也称为GTC 层）是GPON 的核心层，主要完成上行业务流的媒质接入控制和ONU 注册这两个关键功能。GTC 层包括两个子层：GTC 成帧子层和TC 适配子层。 1）GTC帧子层 GTC 成帧子层包括3个功能： 复用和解复用。PLOAM 和GEM 部分根据帧头指示的边界信息复用到下行TC 帧中，并可以根据帧头指示从上行TC 帧中提取出PLOAM 和GEM 部分。 帧头生成和解码。下行帧的TC 帧头按照格式要求生成，上行帧的帧头会被解码。此外还要完成嵌入式OAM。 1、ZigBee协议栈简介 本节内容仅仅是对ZigBee协议栈的一些大家必须理解清楚的概念进行简单的讲解，并没有对ZigBee协议栈的构成及工作原理进行详细的讨论。让刚接触ZigBee协议栈的朋友们对它有个初步的感性认识，有助于后面使用ZigBee协议栈进行真正的项目开发。 什么是ZigBee协议栈呢？它和ZigBee协议有什么关系呢 协议是一系列的通信标准，通信双方需要共同按照这一标准进行正常的数据发射和接收。协议栈是协议的具体实现形式，通俗点来理解就是协议栈是协议和用户之间的一个接口，开发人员通过使用协议栈来使用这个协议的，进而实现无线数据收发。 图1展示了ZigBee无线网络协议层的架构图。ZigBee的协议分为两部分，IEEE 802.15.4定义了PHY（物理层）和MAC（介质访问层）技术规范；ZigBee 联盟定义了NWK（网络层）、APS（应用程序支持子层）、APL（应用层）技术规范。ZigBee协议栈就是将各个层定义的协议都集合在一直，以函数的形式实现，并给用户提供API(应用层)，用户可以直接调用。 图1 ZigBee无线网络协议层 在开发一个应用时，协议较底下的层与应用是相互独立的，它们可以从第三方来获得，因此我们需要做的就只是在应用层进行相应的改动。 介绍到这里，大家应该清楚协议和协议栈的关系了吧，是不是会想着怎么样才能用协议栈来开发自己的项目呢？技术总是不断地在发展地，我们可以用ZigBee厂商提供的协议栈软件来方便地使用ZigBee协议栈（注意：不同厂商提供的协议栈是有区别的，此处介绍TI推出的ZigBee 2007协议栈也称Z-Stack）。 Z-stack是挪威半导体公司Chipcon(目前已经被TI公司收购)推出其CC2430开发平台时，推出的一款业界领先的商业级协议栈软件，由于这个协议栈软件的出现，用户可以很容易地开发出具体的应用程序来，也就是大家说的掌 BLE协议栈简介 协议是一系列的通信标准，通信双方需要共同按照这一标准进行正常的数据发射和接收。协议栈是协议的具体实现形式，通俗点来理解就是协议栈是协议和用户之间的一个接口，开发人员通过使用协议栈来使用这个协议的，进而实现无线数据收发。 下图为BLE协议栈的结构框图： 图 3.1BLE栈架构 协议栈包括两个部分:控制器和主机。控制器和主机在标准蓝牙BR/EDR 设备这两个部分通常是单独实现。任何配置文件和应用程序都是建立在GAP和 GATT协议层上。 ·PHY层：1Mbps自适应跳频GFSK（高斯频移键控），运行在免证的2.4GHz 频段。 ·LL层：RF控制器，控制设备处于准备（standby）、广播（advertising）、监听/扫描（scanning）、初始化（initiating）、连接（connected）这五种状态中一种。 ·HCI层：为接口层，向上为主机提供软件应用程序接口（API），对外为外部硬件控制接口，可以通过串口、SPI、USB来实现设备控制。 ·L2CAP层：为上层提供数据封装服务，允许逻辑上的端到端数据通信。 ·SM层：提供配对和密匙分发服务，实现安全连接和数据交换。 ·GAP层：直接与应用程序或配置文件（profiles）通信的接口，处理设备发现和连接相关服务。另外还处理安全特性的初始化。 ·ATT层：导出特定的数据（称为属性）到其他设备。 ·GATT层：定义了使用ATT的服务框架和配置文件（profiles）的结构。BLE中所有的数据通信都需要经过GATT。 TI的这款CC2540器件可以单芯片实现BLE蓝牙协议栈结构图的所有组件，包括应用程序。 通过上面的介绍，我们基本了解了BLE协议栈的各层功能，其中需要我们直接接触的主要是GAP和GATT这两个层。 编号：_______________本资料为word版本，可以直接编辑和打印，感谢您的下载 TIZstack协议栈开发环境和工作流程 甲方：___________________ 乙方：___________________ 日期：___________________ TI Z-stack 协议栈开发环境和工作流程 系统软件设计是在硬件设计的基础上进行的，良好的软件设计是 实现 系统功能的重要环节，也是提高系统性能的关键所在。节点设计基丁通用 性及便丁开发的考虑，移植了 TI 公司的Z-Stack 协议栈，其主要特点就是其兼 容性，完全支持IEEE 802. 15. 4/ZigBee 的CC2430片上系统解决方案。Z-Sta ck 还支持丰富的新特性，如无线下载，可通过 ZigBee 网状网络(Mesh Networ k)下载节点更新。 Z furn gr irrc vrivci t Bfei AS JJiflll-ltC -liNri l - lie *4--? U ■Bl.K II 图ZigBee 节点开发环境 TI 的Z-Stack 装载在一个基丁 IAR 开发环境的工程里。强大的IAR E mbedded Workbench 除了提供编译下载功能外，还可以结合编程器进行单步跟踪 调试和监测片上寄存器、Flash 数据等。Z-Stack 根据IEEE 802. 15.4 和ZigB ee 标准分为以下几层： API (Application Programming Interface ) , HAL (H ardware Abstract Layer ) , MAC( Media Access Control) , NWK( Zigbee Ne twork Layer ) , OSAL( Operating System Abstract System ) , Security , Se rvice , ZDO(Zigbee Device Objects )。使用 IAR 打开工程文件 SampleApp.e ww 后，即可查看到整个协议栈从 HAL 层到APP 层的文件夹分布。该协议栈可以 实现复杂的网络链接，在协调器节点中实现对路由表和绑定表的非易失性存储， 因此网络具有一定的记忆功能。 Z-Stack 采用操作系统的思想来构建，采用事件轮循机制，当各层 初始 化之后，系统进入低功耗模式，当事件发生时，唤醒系统，开始进入中断 Ri-TPr — iw"" t ■ W T ■ E M ± -ZigBee协议栈OSAL介绍
2020年Zigbee协议栈中文说明免费
7号信令协议栈
Zigbee协议栈原理基础
arp协议栈
蓝牙协议栈BlueZ的移植与开发
CI 协议栈简介
Z-stack协议栈开发
由浅入深,蓝牙4.0BLE协议栈开发攻略大全
协议栈工作原理介绍
协议栈开发总结范文
ResipRocate协议栈(looking)
OMCI协议 介绍 中文版
1.ZigBee协议栈简介
BLE协议栈简介
TIZstack协议栈开发环境和工作流程