Daily Bandwidth Report

Basics

• 2014/03 => The old bandwidth reporting script has been rewritten in the Perl programming language and should now do a better job at keeping up with nwtel's new and improved internet packages.
• The bandwidth report, as installed on a nwtel-connected firewall and configured by Computerisms, will compile a report and send it every night at midnight
• It is important to note that Accounted data should not be used as a measure against what nwtel will charge you
  • It will accurately track how much bandwidth crosses the firewall, but may count more or less than the actual data that crosses your external ethernet port
    • It is a good tool to use for identifying abusers, runaway connections, and other network problems, but does not accurately reflect what nwtel counts
  • The Raw Data should be an accurate count of how much data nwtel will see you using
• When you start using this report, the initial counts will be wrong, how wrong is pretty random, depending on the existing stats on your firewall
  • Daily counts should start being accurate for the 2nd report
  • MTD (Month to Date) counts will start being accurate at the beginning of the next month

FAQ

• Q: Why does the report sometime show me a monthly usage percentage number in the millions or billions?
• A: Short answer: a bug in the script. When the firewall reboots, it's counters start from zero, but the script does not. This creates a mathematical wobble. While the percentage calculation is off, the only usage number that will be incorrect is the number on the day when the firewall rebooted. So whatever your report says for the individual computers, and the actual data used in a given day will still be correct. At the beginning of each month, everything sets back to zero, so the count will start properly again on the 1st of the next month.
• Q: When will this be fixed??
• A: This script took several days to write and was done for 2 hours of pay. Currently, a good number of people use this script, and it has served more than one customer benefit. When a customer sponsors a couple more hours of work on the script, I will fix this bug, along with a few others that have been reported over time...

• Q: What is the difference between Raw data and Accounted data?
• A: Raw data is a measure of bits as they transfer to or from the wire to your Ethernet port, Accounted data is data that has traversed a set of firewall rules
  • Raw Data is counted by your Ethernet hardware driver
    • This data can be differentiated as outgoing or incoming, but no information about the data packets is collected
      • Because no data is collected, it is not possible to tell from this data which of your internal computers generated it
    • This is a measure of the number of bits that pass along your Ethernet cable, and therefore is a measure of the number of bits your nwtel modem will see
  • Accounted data is counted by software known as IPTables
    • Once data comes through the Ethernet port it is transferred to the operating system on your firewall
    • When the operating system receives bits from the Ethernet port, it uses the IPTables software to determine what to do with the data
    • IPTables is capable of inspecting the data's source and destination
      • Because IPTables can collect information, we can determine which internal computer generated the traffic and maintain statistics
    • IPTables uses an ordered series of rules to determine whether to allow or block the data
      • This, along with some helpful tricks from the operating system, is what allows multiple computers to share one Internet connection
    • The stage between IPTables deciding whether to accept the data and whether to send the data is where the data is Accounted
      • Therefore IPTables may count data being sent from or to a computer, but later reject it so it does not show up as Raw data on the wire
    • Once IPTables has decided it is okay to send the data, it passes it back to the operating system, which then puts it on the appropriate Ethernet port for transmission to the wire.
    • One exception to IPTables data collection is the total usage on the external port
      • This is a count of all data leaving IPTables independent of source or destination, and as such cannot determine direction
      • More accurately, it is data from anywhere to anywhere, counted at the last rule as data passes from IPTables back to the operating system

• Q: Why is the Raw and Accounted data on the external port different?
• A: This is because not all data on the wire will make it through IPTables, and not all data that gets Accounted in IPTables will end up on the external wire
  • In most cases Raw data will the higher number, this is because packets show up on the wire that IPTables will not accept for delivery to the internal network.
    • Such packets will include traffic from other devices connected to nwtel's network
      • This is normal and expected and is required to be there, all devices connected to a network must chat with each other to determine who is on which wire so that data arrives at its intended destination
    • Such packets will also include unsolicited transmissions from the internet
      • This includes port scans and other attempts to determine if your firewall is a vulnerable device available for hacking
      • This may also include misconfigured devices on the Internet that think a different devices is at your location
  • In some cases the Accounted data is higher
    • This is because IPTables is transferring data to the operating system that is not destined for the external wire
    • This will happen most frequently on firewalls that connect with multiple internal subnets or with vpns.