# bufferbloat, how to fix?



## orlywutlol

I have horrible downstream bufferbloat (3000ms+) and from what I have read the only way for me to alleviate the issue for the time being is to limit my bandwidth slightly lower than my provisioned speeds provided by my ISP. My router, Netgear N900 WNDR4500 only has QoS for upstream, not downstream. Unless I missed the settings? Enabling QoS for the upstream resulted in no effect. Short of buying a new router or flashing my router with DD-WRT which I have researched and may have issues with bandwidth higher than 100mbps and loss of wifi performance, is there anything else I can do to try and fix this?


----------



## tompsonn

You can only *shape* what you send (i.e. egress, which is why you have the shaping settings for upstream), other than that you need ingress policing to drop packets coming in too fast. Most home routers can't do this, but a Linux box surely can.

You are correct that the solution is to limit the bandwidth to somewhat lower than the maximum speed to prevent buffering at the ISP (for downstream) or the modem (for upstream).

Here is the Wonder Shaper script you can use on a Linux box to achieve this:

Code:



Code:


#!/bin/bash
# Wonder Shaper
# please read the README before filling out these values 
#
# Set the following values to somewhat less than your actual download
# and uplink speed. In kilobits. Also set the device that is to be shaped.

DOWNLINK=800
UPLINK=220
DEV=ppp0

# low priority OUTGOING traffic - you can leave this blank if you want
# low priority source netmasks
NOPRIOHOSTSRC=

# low priority destination netmasks
NOPRIOHOSTDST=

# low priority source ports
NOPRIOPORTSRC=

# low priority destination ports
NOPRIOPORTDST=

# Now remove the following two lines :-)

echo Please read the documentation in 'README' first
exit

if [ "$1" = "status" ]
then
        tc -s qdisc ls dev $DEV
        tc -s class ls dev $DEV
        exit
fi

# clean existing down- and uplink qdiscs, hide errors
tc qdisc del dev $DEV root    2> /dev/null > /dev/null
tc qdisc del dev $DEV ingress 2> /dev/null > /dev/null

if [ "$1" = "stop" ] 
then 
        exit
fi

###### uplink

# install root HTB, point default traffic to 1:20:

tc qdisc add dev $DEV root handle 1: htb default 20

# shape everything at $UPLINK speed - this prevents huge queues in your
# DSL modem which destroy latency:

tc class add dev $DEV parent 1: classid 1:1 htb rate ${UPLINK}kbit burst 6k

# high prio class 1:10:

tc class add dev $DEV parent 1:1 classid 1:10 htb rate ${UPLINK}kbit \
   burst 6k prio 1

# bulk & default class 1:20 - gets slightly less traffic, 
# and a lower priority:

tc class add dev $DEV parent 1:1 classid 1:20 htb rate $[9*$UPLINK/10]kbit \
   burst 6k prio 2

tc class add dev $DEV parent 1:1 classid 1:30 htb rate $[8*$UPLINK/10]kbit \
   burst 6k prio 2

# all get Stochastic Fairness:
tc qdisc add dev $DEV parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10
tc qdisc add dev $DEV parent 1:30 handle 30: sfq perturb 10

# TOS Minimum Delay (ssh, NOT scp) in 1:10:

tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \
      match ip tos 0x10 0xff  flowid 1:10

# ICMP (ip protocol 1) in the interactive class 1:10 so we 
# can do measurements & impress our friends:
tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \
        match ip protocol 1 0xff flowid 1:10

# To speed up downloads while an upload is going on, put ACK packets in
# the interactive class:

tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
   match ip protocol 6 0xff \
   match u8 0x05 0x0f at 0 \
   match u16 0x0000 0xffc0 at 2 \
   match u8 0x10 0xff at 33 \
   flowid 1:10

# rest is 'non-interactive' ie 'bulk' and ends up in 1:20

# some traffic however suffers a worse fate
for a in $NOPRIOPORTDST
do
        tc filter add dev $DEV parent 1: protocol ip prio 14 u32 \
           match ip dport $a 0xffff flowid 1:30
done

for a in $NOPRIOPORTSRC
do
        tc filter add dev $DEV parent 1: protocol ip prio 15 u32 \
           match ip sport $a 0xffff flowid 1:30
done

for a in $NOPRIOHOSTSRC
do
        tc filter add dev $DEV parent 1: protocol ip prio 16 u32 \
           match ip src $a flowid 1:30
done

for a in $NOPRIOHOSTDST
do
        tc filter add dev $DEV parent 1: protocol ip prio 17 u32 \
           match ip dst $a flowid 1:30
done

# rest is 'non-interactive' ie 'bulk' and ends up in 1:20

tc filter add dev $DEV parent 1: protocol ip prio 18 u32 \
   match ip dst 0.0.0.0/0 flowid 1:20

########## downlink #############
# slow downloads down to somewhat less than the real speed  to prevent 
# queuing at our ISP. Tune to see how high you can set it.
# ISPs tend to have *huge* queues to make sure big downloads are fast
#
# attach ingress policer:

tc qdisc add dev $DEV handle ffff: ingress

# filter *everything* to it (0.0.0.0/0), drop everything that's
# coming in too fast:

tc filter add dev $DEV parent ffff: protocol ip prio 50 u32 match ip src \
   0.0.0.0/0 police rate ${DOWNLINK}kbit burst 10k drop flowid :1

http://www.lartc.org/wondershaper/


----------



## orlywutlol

Quote:


> Originally Posted by *tompsonn*
> 
> You can only *shape* what you send (i.e. egress, which is why you have the shaping settings for upstream), other than that you need ingress policing to drop packets coming in too fast. Most home routers can't do this, but a Linux box surely can.
> 
> You are correct that the solution is to limit the bandwidth to somewhat lower than the maximum speed to prevent buffering at the ISP (for downstream) or the modem (for upstream).
> 
> Here is the Wonder Shaper script you can use on a Linux box to achieve this:
> 
> http://www.lartc.org/wondershaper/


This sounds interesting.. so how would I set up something like this? I need a separate computer running Linux?


----------



## Derp

Are you using dslreport's new test? Capping my bandwidth with Tomato does help with bloat.

Uncapped graded F:

Idle = 72ms
DL = 52.8 megabit/s = 1282ms
UL = 5.97 megabit/s = 955ms

Capped in Tomato graded B:

Idle = 71ms
DL = 48 megabit/s = 87ms
UL = 4.25 megabit/s = 185ms


----------



## tompsonn

Quote:


> Originally Posted by *orlywutlol*
> 
> This sounds interesting.. so how would I set up something like this? I need a separate computer running Linux?


Yes, or perhaps some custom firmware on your router may support this.


----------



## richbhanover

Quote:


> Originally Posted by *orlywutlol*
> 
> I have horrible downstream bufferbloat (3000ms+) and from what I have read the only way for me to alleviate the issue for the time being is to limit my bandwidth slightly lower than my provisioned speeds provided by my ISP. My router, Netgear N900 WNDR4500 only has QoS for upstream, not downstream. Unless I missed the settings? Enabling QoS for the upstream resulted in no effect. Short of buying a new router or flashing my router with DD-WRT which I have researched and may have issues with bandwidth higher than 100mbps and loss of wifi performance, is there anything else I can do to try and fix this?


It isn't clear that Wondershaper is going to do what you need - there's an essay that describes its failings at: http://www.bufferbloat.net/projects/cerowrt/wiki/Wondershaper_Must_Die

And whether it did or not, attempting to use QoS and other shaping policies for this introduces maintenance hassles, as you try to balance the priorities of various kinds of traffic.

Regrettably, your router (WNDR4500) doesn't seem to be supported by either DD-WRT or OpenWrt (it's a work in progress - wip - for both). If it did, the SQM/fq_codel facility would give a straightforward fix.

fq_codel works by reclaiming control of the bottleneck link - in both directions. When you set it for a couple percent below the full-speed for each direction, it prevents a queue of data from building up for any of the flows/connections at any time. For details, see http://www.bufferbloat.net/projects/codel/wiki/Technical_description_of_FQ_CoDel


----------



## orlywutlol

Derp:
I am. And this is what I would like to do. I am trying to find the easiest way to cap my download since my bufferbloat is on the download side. I tried doing this with my router but it appears it doesn't support the features that I need. Comcast is my ISP and apparently they are aware of the issue but there is no solution on their end.

richbhanover:
I found a relatively in depth and somewhat active thread about using DD-WRT on my WNDR4500 found here. I am doing some research on this to see if this will be the best solution for me. I have read that using QoS with my router caps bandwidth around 100mbps, limited by cpu of the router? And I am not even sure if doing this will give me the desired outcome.

I am wiling to spend some money to come up with a solution that doesn't half ass it. Whether that be flashing my router with DD-WRT, buy a new router that supports fq_codel? If that is even possible? a pfense build? I came across this that might be something to look into. I am not sure which direction to proceed..


----------



## dtaht

oy. I have been trying to get wondershaper fixed in the field for nearly a decade now. Don't use it as is, please, at higher rates.

full description of all the problems in it: http://www.bufferbloat.net/projects/cerowrt/wiki/Wondershaper_Must_Die

in particular the default inbound policer is broken for ipv6 entirely, and has a burst size suitable for 2002 era bandwidths.


----------



## orlywutlol

I was doing some research and decided I wanted to try pfsense out instead of trying to flash my wndr4500 to DD-WRT. I came across a neat little mini book size pc on newegg. Build is as follows:

OEM Production 2550L2D-MxPC Intel NM10 2 x 204Pin SO-DIMM Intel GMA 3650 $129.99

SanDisk ReadyCache SDSSDRC-032G-G26 2.5" 32GB SATA III $35.99

G.SKILL 4GB (2 x 2GB) 204-Pin DDR3 SO-DIMM DDR3 1066 (PC3 8500) $44.99

For a total of $210.97 not including shipping. What are your thoughts? Drop down to 2gb ram? Is there anything comparable thats cheaper/more features?


----------



## Archea47

pfsense is cool and I think you're heading the right route

Also consider Fortinet; their SMB line is priced very reasonably and one of their UTM features is traffic shaping


----------

