輸入方向的流量控制

1、概述 Linux中的QoS分為入口(Ingress)部分和出口(Egress)部分，入口部分主要用于進行入口流量限速(policing)，出口部分主要用于隊列調(diào)度(queuing scheduling)。大多數(shù)排隊規(guī)則(qdisc)都是用于輸出方向的，輸入方向只有一個排隊規(guī)則，即ingress qdisc。ingress qdisc本身的功能很有限，但可用于重定向incoming packets。通過Ingress qdisc把輸入方向的數(shù)據(jù)包重定向到虛擬設(shè)備ifb，而ifb的輸出方向可以配置多種qdisc，就可以達到對輸入方向的流量做隊列調(diào)度的目的。 Q: 為什么大多數(shù)的

2、流量控制都是在輸出方向的？A: It is easiest to create traffic control rules for traffic flowing out of an interface, since we can control when the system    sends data, but controlling when we receive data requires an additional intermediate queue to be created to buffer    inco

3、ming data. 原理圖如下：  Ingress qdisc The ingress qdisc itself does not require any parameters. It differs from other qdiscs in that it does not occupy theroot of a device. Attach it like this:# tc qdisc add dev eth0 ingressThis allows you to have other, sending qdiscs on your device

4、besides the ingress qdisc. About the ingress qdiscIngress qdisc (known as ffff:) can't have any children classes. (hence the existence of IMQ)The only thing you can do with the ingress qdisc is attach filters. About filtering on the ingress qdiscSince there are no classes to which to d

5、irect the packets, the only reasonable option is to drop the packets.With clever use of filtering, you can limit particular traffic signatures to particular uses of your bandwidth. 入口流量的限速# tc qdisc add dev eth0 ingress# tc filter add dev eth0 parent ffff: protocol ip prio 10 u32 match ip src 0

6、.0.0.0/0 police rate 2048kbps burst 1m drop flowid :1 ifb IFB Intermediate Functional Block device。Q: How can we use qdisc (e.g., netem) on incoming traffic?A: You need to use IFB. This network device allows attaching queueing disciplines to incoming packets.To use an IFB, you must have IF

7、B support in your kernel (configuration option CONFIG_IFB). Assuming thatyou have a modular kernel, the name of the IFB module is ifb and may be loaded using the commandmodprobe ifb (if you have modprobe installed) or insmod /path/to/module/ifb.ip link set ifb0 upip link set ifb1 upBy default, two I

8、FB devices(ifb0 and ifb1) are created. IFB allows for queueing incoming traffic for shaping instead of dropping. ifb模塊需要手動加載。# modprobe ifb啟用虛擬設(shè)備ifb0。 # ip link set dev ifb0 up使用ifb0做輸入方向的重定向。tc filter add dev eth0 parent fff: protocol ip u32 match u32 0 0 flowid 1:1 action mirred egr

9、ess redirect dev ifb0使用ifb0做輸出方向的重定向。tc filter add dev eth0 parent 1: protocol ip u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb0 實例 用ingress qdisc和ifb做ingress方向的隊列調(diào)度。# modprobe ifb# ip link set dev ifb0 up txqueuelen 1000# tc qdisc add dev eth1 ingress# tc filter add d

10、ev eth1 parent ffff: protocol ip u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb0# tc qdisc add dev ifb0 root netem delay 50ms loss 1% Author zhangskd csdn Reference 1  /howto/index.html 2  /collaborate

11、/workgroups/networking/ifbThe Intermediate Functional Block device is the successor to the IMQ iptables module that was never integrated. Advantage over current IMQ; cleaner in particular in SMP; with a _lot_ less code. Old Dummy device functionality is preserved while new one only

12、60;kicks in if you use actions.To use an IFB, you must have IFB support in your kernel (configuration option CONFIG_IFB). Assuming that you have a modular kernel, the name of the IFB module is 'ifb' and may be loaded using the command modprobe ifb (if you have modprobe installed) o

13、r insmod /path/to/module/ifb.  ip link set ifb0 up ip link set ifb1 upBy default, two IFB devices (ifb0 and ifb1) are createdIFB UsageAs far as i know the reasons listed below is why people use IMQ. It would be nice to know of anything else that i missed.· qdiscs/polici

14、es that are per device as opposed to system wide. IMQ allows for sharing.· Allows for queueing incoming traffic for shaping instead of dropping. I am not aware of any study that shows policing is worse than shaping in achieving the end goal of rate control. I would be interested if anyone is ex

15、perimenting. (re shaping vs policing: the desire for shaping comes more from the need to have complex rules like with htb)· Very interesting use: if you are serving p2p you may wanna give preference to your own localy originated traffic (when responses come back) vs someone using your system to

16、 do bittorent. So QoSing based on state comes in as the solution. What people did to achieve this was stick the IMQ somewhere prelocal hook. I think this is a pretty neat feature to have in Linux in general. (i.e not just for IMQ).But I wont go back to putting netfilter hooks in the device to satisf

17、y this. I also dont think its worth it hacking ifb some more to beaware of say L3 info and play ip rule tricks to achieve this.Instead the plan is to have a contrack related action. This action will selectively either query/create contrack state on incoming packets. Packets could then be redirected

18、to ifb based on what happens (e.g. on incoming packets); if we find they are of known state we could send to a different queue than one which didnt have existing state. This all however is dependent on whatever rules the admin enters.At the moment this function does not exist yet. I have decided ins

19、tead of sitting on the patch to release it and then if theres pressure i will add this feature.What you can do with ifb currently with actionsWhat you can do with ifb currently with actionsLets say you are policing packets from alias 00/32 you dont want those to exceed 100kbps

20、 going out.tc filter add dev eth0 parent 1: protocol ip prio 10 u32    match ip src 00/32 flowid 1:2   action police rate 100kbit burst 90k dropIf you run tcpdump on eth0 you will see all packets going out with src 00/32 dropped or no

21、tExtend the rule a little to see only the ones that made it out:tc filter add dev eth0 parent 1: protocol ip prio 10 u32  match ip src 00/32 flowid 1:2   action police rate 10kbit burst 90k drop  action mirred egress mirror dev ifb0 Now fire tcpdump on ifb0

22、 to see only those packets .tcpdump -n -i ifb0 -x -e -t Essentially a good debugging/logging interface.If you replace mirror with redirect, those packets will be blackholed and will never make it out. This redirect behavior changes with new patch (but not the mirror).IFB ExampleMany readers hav

23、e found this page to be unhelpful in terms of expressing how IFB is useful and how it should be used usefully.These examples are taken from a posting of Jamal at http:/www.mail-What this script will demonstrate is the following sequence:    any packet coming going out on eth0 10.

24、0.0.229 is classified as class 1:10 and redirected to ifb0.        on reaching ifb0 the packet is classified as class 1:2        subjected to a token buffer shaping of rate 20kbit/s        sent b

25、ack to eth0    on coming back to eth0, the classificaction 1:10 is still valid and this packet is put through an HTB classifier which limits the rate to 256KbpsWhat this script will demonstrate is the following sequence:1) any packet coming going out on eth0 29 is classified a

26、sclass 1:10 and redirected to ifb0.2) a) on reaching ifb0 the packet is classified as class 1:2   b) subjected to a token buffer shaping of rate 20kbit/s   c) sent back to eth03) on coming back to eth0, the classificaction 1:10 is still validand this packet is put through an HTB

27、classifier which limits the rateto 256Kbpsexport TC="/sbin/tc"$TC qdisc del dev ifb0 root handle 1: prio$TC qdisc add dev ifb0 root handle 1: prio$TC qdisc add dev ifb0 parent 1:1 handle 10: sfq$TC qdisc add dev ifb0 parent 1:2 handle 20: tbf rate 20kbit buffer 1600 limit 3000$TC qdisc add

28、 dev ifb0 parent 1:3 handle 30: sfq                               $TC filter add dev ifb0 parent 1: protocol ip prio 1 u32 match ip dst 11.0

29、.0.0/24 flowid 1:1$TC filter add dev ifb0 parent 1: protocol ip prio 2 u32 match ip dst /24 flowid 1:2ifconfig ifb0 up$TC qdisc del dev eth0 root handle 1: htb default 2$TC qdisc add dev eth0 root handle 1: htb default 2$TC class add dev eth0 parent 1: classid 1:1 htb rate 800

30、Kbit$TC class add dev eth0 parent 1: classid 1:2 htb rate 800Kbit$TC class add dev eth0 parent 1:1 classid 1:10 htb rate 256kbit ceil 384kbit$TC class add dev eth0 parent 1:1 classid 1:20 htb rate 512kbit ceil 648kbit$TC filter add dev eth0 parent 1: protocol ip prio 1 u32 match ip dst 

31、29/32 flowid 1:10 action mirred egress redirect dev ifb0A Little test (be careful if you are sshed in and are classifying onthat IP, counters may be not easy to follow)-A ping .mambo:# ping -c2 29/ first at ifb0/ observe that second filter twice being successfulmambo:# $TC -s filter sho

32、w dev ifb0 parent 1:filter protocol ip pref 1 u32filter protocol ip pref 1 u32 fh 800: ht divisor 1filter protocol ip pref 1 u32 fh 800:800 order 2048 key ht 800 bkt 0 flowid1:1  (rule hit 2 success 0)  match 0b000000/ffffff00 at 16 (success 0 )filter protocol ip pref 2 u32filter protocol

33、ip pref 2 u32 fh 801: ht divisor 1filter protocol ip pref 2 u32 fh 801:800 order 2048 key ht 801 bkt 0 flowid1:2  (rule hit 2 success 2)  match 0a000000/ffffff00 at 16 (success 2 )/next the qdisc numbers ./Observe that 1:2 has 2 packetsmambo:# $TC -s qdisc show dev ifb0qdisc prio 1: bands

34、3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1Sent 196 bytes 2 pkt (dropped 0, overlimits 0 requeues 0)rate 0bit 0pps backlog 0b 0p requeues 0qdisc sfq 10: parent 1:1 limit 128p quantum 1514bSent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)rate 0bit 0pps backlog 0b 0p requeues 0qdisc tbf 20:

35、parent 1:2 rate 20000bit burst 1599b lat 546.9msSent 196 bytes 2 pkt (dropped 0, overlimits 0 requeues 0)rate 0bit 0pps backlog 0b 0p requeues 0qdisc sfq 30: parent 1:3 limit 128p quantum 1514bSent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)rate 0bit 0pps backlog 0b 0p requeues 0/ Next look a

36、t eth0/ observe class 1:10 which is where the pings went through after/ they came back from the ifb0 device.mambo:# $TC -s class show dev eth0class htb 1:1 root rate 800000bit ceil 800000bit burst 1699b cburst 1699bSent 196 bytes 2 pkt (dropped 0, overlimits 0 requeues 0)rate 0bit 0pps backlog 0b 0p

37、 requeues 0lended: 0 borrowed: 0 giants: 0tokens: 16425 ctokens: 16425class htb 1:10 parent 1:1 prio 0 rate 256000bit ceil 384000bit burst 1631bcburst 1647bSent 196 bytes 2 pkt (dropped 0, overlimits 0 requeues 0)rate 0bit 0pps backlog 0b 0p requeues 0lended: 2 borrowed: 0 giants: 0tokens: 49152 cto

38、kens: 33110class htb 1:2 root prio 0 rate 800000bit ceil 800000bit burst 1699b cburst 1699bSent 47714 bytes 321 pkt (dropped 0, overlimits 0 requeues 0)rate 3920bit 3pps backlog 0b 0p requeues 0lended: 321 borrowed: 0 giants: 0tokens: 16262 ctokens: 16262class htb 1:20 parent 1:1 prio 0 rate 512000b

39、it ceil 648000bit burst 1663bcburst 1680bSent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)rate 0bit 0pps backlog 0b 0p requeues 0lended: 0 borrowed: 0 giants: 0tokens: 26624 ctokens: 21251-mambo:# $TC -s filter show dev eth0 parent 1:filter protocol ip pref 1 u32filter protocol ip pref 1 u32 f

40、h 800: ht divisor 1filter protocol ip pref 1 u32 fh 800:800 order 2048 key ht 800 bkt 0 flowid1:10  (rule hit 235 success 4)  match 0a0000e5/ffffffff at 16 (success 4 )        action order 1: mirred (Egress Redirect to device ifb0) stolen   

41、;     index 2 ref 1 bind 1 installed 114 sec used 100 sec        Action statistics:        Sent 196 bytes 2 pkt (dropped 0, overlimits 0 requeues 0)        rate 0bit 0pps back

42、log 0b 0p requeues 0IFB requirementsIn order to use ifb you need:    Support for ifb on kernel (2.6.20 works OK)        Menu option: Device drivers -> Network device support -> Intermediate Functional Block support    &

43、#160;   Module name: ifb    Tc iproute2 with support of "actions" (2.6.20 - 20070313 works OK and package from Debian etch is outdated). You can download it from here: /dev/iproute2/download/Ingress qdiscAll qdiscs discussed so far are

44、egress qdiscs. Each interface however can also have an ingress qdisc which is not used to send packets out to the network adaptor. Instead, it allows you to apply tc filters to packets coming in over the interface, regardless of whether they have a local destination or are to be forwarded.As the tc

45、filters contain a full Token Bucket Filter implementation, and are also able to match on the kernel flow estimator, there is a lot of functionality available. This effectively allows you to police incoming traffic, before it even enters the IP stack.14.4.1. Parameters & usageThe ingress qdisc it

46、self does not require any parameters. It differs from other qdiscs in that it does not occupy the root of a device. Attach it like this:# delete originaltc qdisc del dev eth0 ingresstc qdisc del dev eth0 root# add new qdisc and filtertc qdisc add dev eth0 ingresstc filter add dev eth0 parent ffff: protocol ip prio 50  u32 match ip src /0 police rate 2048kbps burst 1m drop flowid :1tc qdisc add dev eth0 root tbf rate 2048kbps latency 50ms burst 1mI played a bit with the ingress qdisc after seeing Patric