-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

   _   _ _
  | \ | | |_ ___  _ __
  |  \| | __/ _ \| '_ \
  | |\  | || (_) | |_) |
  |_| \_|\__\___/| .__/
                 |_|

             Network Top

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
                        Fully revised for ntop 3.0
                        Updated for ntop 3.1 and 3.2
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Compiled by Burton M. Strauss III <Burton@ntopsupport.com> - comments to him!
In this document, that's whom "I" or "My" refers to.

      |\      _,,,---,,_       "If I purr, you must have fed me.
      /,`.-'`'    -.  ;-;;,_    If the network purrs, it must
    <|,4-  ) )-,_..;\ (  `'-'   be ntop!"   -----Pixel
      '---''(_/--'  `-'\_)

As usual, entries are in no particular sequence.  However, there is an
overall flow, starting with very general questions, then questions about
specific features, then questions about specific platforms.

What structure there is in this FAQ is that of a dialog with a bright 7 year
old.  The response to every answer is "Why".  Keep reading until you get
too detailed for your needs.

If you actually read through this FAQ and don't find yourself saying "Oh,
Bother", at least twice, and then go read Winnie the Pooh.

At the very end are "HowTo Ask For Help" and a "GDB ultraMini-tutorial".
Read them.  Before posting to the mailing list.


Note that some of this FAQ is based on the entries people contributed
to the community FAQ formerly at http://snapshot.ntop.org.  Lots more are 
based on messages to the ntop and ntop-dev mailing lists.


--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
TOP 10 - the questions everyone asks...

Q0. I downloaded the source and ./configure doesn't work.
A. Yup.

   There is a long history of warfare between the versions of the GNU
   autotools we use to build the distribution files and the ones installed
   on your host. So during the 3.3 development cycle, Luca finally gave in
   and stopped distributing a generated configure file.

   Instead there is a script, autogen.sh, which uses the version(s) of the
   tools installed on your system to create configure and then run it.

   Yes, this means you MUST have the following tools installed:

      libtool
      automake
      m4
      autoconf

   We are talking only about people who are compiling from source.  Those 
   tools are pretty much required for ANY source.  Live with it.


Q1(a). Can I store data in a SQL database?
Q1(b). When ntop stops I lose all my data.  Why?
Q1(c). Why doesn't the -S option work?

A. ntop used to optionally store some data in a SQL database.  The code was
   broken, difficult to maintain, etc. and was removed.  A LONG TIME AGO.
   If you are reading about this in 'some' documentation - update.

   Current ntop is 3.1, which is the only version we support.

   There are scripts that various users have offered to take the data dump
   and insert it into a SQL database.  Search the back traffic on the mailing
   list for them.

   Yes, ntop uses memory based structures to hold usage data and they are lost
   when you reset or restart ntop.

   Persistent storage is in the RRD databases - there's a paper @ SourceForge
   that explains them.

   There was another option for some persistence - it was -S - look in FAQarchive 
   for an article about it, "What was the -S option?".


Q2. The archive isn't indexed, so I can't search it.
A. Yes, but it's easy to search using search engines or mail archives.

   Google: You need to restrict the search to the U of Pisa mail list
   gateway, i.e.:

        site:listgateway.unipi.it ntop freebsd

   Two 'mail archive' sites that I know of (as of March 2005) are:

   gmane: http://search.gmane.org (our lists are renamed as gmane.linux.ntop.general
   and gmane.linux.ntop.devel).

   The Mail Archive:

       http://www.mail-archive.com/ntop-dev@unipi.it/
       http://www.mail-archive.com/ntop@unipi.it/


Q3. ntop crashed and the last log message is "xxxxx".
A. Sorry, that's useless.  ntop is multi-threaded and processes 100s or 1000s of
   packets per second.  The last log message is probably from the wrong thread 
   and many seconds out of date.

   To capture the true 'failure point information', you need to run under the 
   debugger (gdb) and send us the various outputs.  Instructions are at the bottom
   of this document.  Look for "GDB ultraMini-tutorial".

   Yes, you will need the source and a compiler.

Q3(a). What about the backtrace?
A. Again, probably useless.  Use gdb and capture the full failure point info. If you
   want to see more, look for "Q. What about the backtrace?" below.


Q4. I'm running out of memory.
A. Basically ntop uses a lot of memory - it stores a chunk of information about each 
   and every host it's monitoring. See "Q. Why does ntop use so much memory ?" and
   the following articles below.


Q5. Dropped packets?
A. There are lots of reasons - look for "Q. Why does ntop drop packets?" below.
   Short version, is that as long as it's random, the information you glean from ntop
   (32% of our usage is P2P Music) is still valid.  But you are probably understating
   any problems.


Q6. The docs at ntop.org say ...
A. Stop right there.  They're out of date (notice, for example, the man page is
   July 2002?).

   The only current stuff is this FAQ (which gets out of date quickly) the other files
   in the distribution and the mailing lists.

   You can access the updated FAQ from docs/FAQ in the cvs or from your ntop instance
   (click on the (?) icon in the "About" menu and read down 1/2 a page or so.


Q7. How do I report a problem?
A. Best way is to use the PR (Problem Report) form - it automatically includes a lot of
   the important information about your ntop instance.  In the "About" menu, click on the
   "Bug" icon.  This generates a text form you can copy into your email program, update
   and send.


Q8. HACK ALERT HACK ALERT - ntop is sending information to some machine called "jake".
A. Chill.  Take a deep breath.

   Jake is the cannonical name for version.ntop.org.  All you are seeing is the version
   check.  See "Q. What's with the version check?" below.


Q9. The program asks for password for "ntop HTTP server". When I started NTOP for the first 
   time, it asked me to set admin password, and i put "xxxxxxxx". The user is "xxxxxxxxxx"
   but I get "Unauthorized to access the document". Why?
A. The correct user to specify for the ntop web server is admin.
   The -u value in the command line or parameter file is the account ntop runs under.


Q10. I'm not getting any rrd output.
A. First, make sure that the plugin is installed (you should see it in the web interface
   under plugins).  Make sure it is configured and active.

   The default configuration does NOT include per-host .rrd files, because these can take
   up a lot of space.  But you probably want them and at the medium or high level of detail.

Q11. Single Threaded ntop?
A. Gone.  Poof.  We're in a multi-threaded world, so live with it.


--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Where can I find ntop?
A. The official website can be found at http://www.ntop.org/.

Q. Where do I get the source?
A. SourceForge -- http://sourceforge.net/projects/ntop/

   There is also a cvs (current development) maintained at cvs.ntop.org.
   (instructions are on the download page of www.ntop.org)


Q. What is ntop?
A. ntop is an open source network top - it monitors a network and collects
   information about the protocols and hosts for display.


Q. Um, so it's like mrtg (http://people.ee.ethz.ch/~oetiker/webtools/mrtg/)?
A. Yes and no...

   Yes in that both are analyzers of network packets.

   Yes in that both display information about your network.

   No in that they take very different approaches to collecting information.

   No in that they display different types of information.


Q. So mrtg...
A. mrtg creates a picture of the network centered on the device on the link
   between devices (and aggregations of devices and links).

   Tobias (Tobi) Oetiker describe mrtg as:

       "The Multi Router Traffic Grapher (MRTG) is a tool to monitor
        the traffic load on network-links. MRTG generates HTML pages
        containing graphical images which provide a LIVE visual
        representation of this traffic."


Q. And mrtg works how?
A. mrtg reads the counters maintained by various devices such as routers, using
   a protocol called 'snmp' (Simple Network Monitoring Protocol).  The
   management information bases (MIBs) read using snmp, contain incredibly
   detailed information about the packets the device has seen and what it has
   done with them.

   Again, quoting Tobi:

       "MRTG ... uses SNMP to read the traffic counters of your routers and
        ... which logs the traffic data and creates beautiful graphs representing
        the traffic on the monitored network connection."

   mrtg is focused on 'layer 2' (the tcp/ip and other low level protocol).

Q. And ntop?
A. ntop doesn't use snmp for it's main analysis - it's not a device centric 
   view of the network.  Instead ntop actually processes the network packets
   directly.


Q. What's wrong with snmp?
A. Nothing.  As of 3.1, ntop has some sort of snmp plugin.

   It's just different.

   snmp is a pull protocol, that is a monitoring tool has to pull the
   information from the device.  snmp has 'traps' that is alert messages which
   can be sent out, but the MIB data has to be read by the monitoring tool from
   the device.

   snmp is an older protocol, from the dawn of the network era and it has some
   minor issues of security and complexity and verbosity.  But it's a critical
   network protocol, used successfully by 1000s of ISPs to monitor AND CONTROL
   vast networks.

   From our perspective, the problems with snmp are minor -

   It can use a lot of bandwidth - especially if you're reading from devices
   on the far side of slow links.

   Pulling data out of MIBs is a complex process. MIBs can be specified in
   an RFC, or be unique to a vendor/device.

   An snmp-based tool either has to restrict itself to the common MIBs,
   or (most often for vendor tools) it be updated whenever a new device
   must be supported or a MIB changes.

   This makes snmp-based tools complex, because data may be unavailable
   in certain versions of seemingly similar devices, etc.  For example,
   http://www.cisco.com/public/sw-center/netmgmt/cmtk/mibs.shtml
   is a page which links to the specific MIBs supported by various
   Cisco devices.


Q. So why hasn't ntop 'won'?
A. mrtg excels at monitoring 100s or 1000s of network devices (routers,
   switches, etc.) and presenting that information over long periods of time.

   ntop doesn't do a good job of showing multiple 'networks' - it's really
   focused on aggregating a picture of a single network.  And for drilling down
   into that picture or presenting it over long periods of time.

   The processing of packets requires a lot more computer resources than just
   reading counters from devices.  On the plus side, this gives much more
   detailed information - for example ntop sees the actual web server request
   instead of just that there was traffic on port 80.  On the minus side, it's
   pretty easy to exceed the processing power of the low end machine typically
   available for ntop.  An ISP using ntop to monitor a couple of T3s needs a
   FAST computer and A LOT of memory.

   ntop also requires access to the physical network (either directly via a
   network card or indirectly via a netFlow/sFlow probe).  This limits ntop's
   (usefullness|ability) to work across sites.

   Once you learn what they do (mrtg and ntop), you'll probably discover that
   You need both.


Q. What's this 'layer' crud?
A. Network layers come from a widely cited but never implemented model, the OSI
   (Open System Interconnect ) networking model from the ISO (International
   Standards Organization).

   Google for it - for example
       http://www.ussg.iu.edu/usail/network/nfs/network_layers.html


Q. So ntop is like netFlow (Cisco), sFlow (RFC 3176, http://www.sflow.org) or
   RMON (HP)?
A. Not really, actually those are all protocols for sending and receiving
   information about the network.

   ntop has lots in common with the tools that USE those protocols.

   And there are lots of tools - some proprietary, many open source.

   The devices/programs that collect the information and send it out in netFlow,
   sFlow or RMON format are usually called (by me) 'probes'.  The devices and/or
   programs that receive the netFlow, sFlow or RMON formatted information and do
   things with it are called 'collectors' (if they process it and forward it on)
   or called 'displays'.

   In fact, ntop can receive netFlow packets and both send and receive sFlow 
   packets.  It can be a 'probe' or a 'display'. (ntop used to be able to send
   netFlow packets - that was removed 2004-03 by Luca).


Q. So ntop is like Nagios or Ipswitch's - WhatsUp Gold?.
A. Nope - those are layer 4 and higher (application) monitoring programs.


Q. So it's like ...
A. Enough already - if you search Freshmeat.net,
   http://freshmeat.net/search/?q=network+monitor&section=projects
   you will find (as of 18Aug2003):


       Topic :: System :: Networking :: Monitoring (654 projects)

   We'll be here all day.  ntop is ntop.


Q. Ok, so ntop is a unique TCP/IP analyzer.
A. Not exactly.

   First off, ntop pretty much doesn't care about the lowest (layer 1 or wire)
   layer.  It leaves dealing with that to a library, libpcap, which hides most
   of that.

   ntop is designed as a hybrid packet analyzer, not a pure Ethernet analyzer
   (layer 2) nor a pure TCP/IP analyzer (layer 3).

   ntop gets the data at the layer 2 (frame) level, which could be Ethernet
   or another protocol.  Beyond Ethernet, ntop has minimal smarts about FDDI,
   PPP, RAW and TOKEN-RING frames.  That is, at least enough for some basic
   counts or to extract the (layer 3) TCP/IP data in side.

   ntop 3.0 adds TWO (three?) HUGE areas of new protocol support.

   ntop 3.0 supports IPv6, thanks to code contributed by Olivier Festor
   <Olivier.Festor@loria.fr> and Abdelkader Lahmadi <Abdelkader.Lahmadi@loria.fr>
   of the MADYNES Research Time (Managing DYnamic NEtworks and Services), see
   http://madynes.loria.fr/.

   ntop 3.0 has more than a fair amount of smarts about FibreChannel and SCSI
   thanks to code contributed by Dinesh G Dutt <ddutt@cisco.com> of Cisco. 

   However, since most of ntop's displayed counts are at the TCP/IP level, it
   confuses people into thinking ntop is purely a TCP/IP analyzer.

   ntop is a traffic monitor with it's own network interfaces, which monitors
   what it sees (or is told about through netFlow or sFlow probes).



Q. ntop runs under?
A. ntop is known to work under Linux, Mac OS X, FreeBSD, Solaris and Win32.

   ntop development is done primarily on Fedora Linux.

   Luca also does a port to Win32 (MS Visual C .Net) and used to work in the Sun
   Solaris environment.  He also seems to work with FreeBSD 5.4.

   I (Burton) usually work under Fedora Linux, but use Microsoft's VPC 2004 and 
   VMware Workstation 5 to test under other Linux distros and OSes.

   Our users run under many other platforms - Here's data from the version.xml log
   records showing what people were running while testing 3.2:

   (This data covered approximately the first two weeks of July 2005)

      count version
    ------- -------
      16684 3.1
      10960 3.0
        345 3.1rc1
        259 3.1.1
    etc.

     count OS Version
   ------- ------- --------------
     20509 Linux
      3566 Windows WinNT/2K/XP
      1752 Unknown Windowsv3.1
      1018 Unknown Windowsv3.0
       266 FreeBSD 5.3
       234 FreeBSD 5.4
       219 Darwin 7.7.0

   Of course, this is self-selected - people can turn off the logging.

   Linux covers lots of Different Linuxes... of the ones we recognize:

     count Distro    Release  
   ------- --------- -------- 
      3148 suse               
      2637 redhat    9        
      2625 fedora    3        
      2014 fedora    2        
      1541 debian    3.1      
      1373 gentoo    1.4.16   
       613 redhat    3        
       603 fedora    1        
       523 mandrakel 10.2     
       505 gentoo    1.6.12   
       473 redhat    7.3      
       448 mandrakel 10.1     
       428 debian             
       399 redhat    4        
       367 redhat    8.0      
       301 slackware 10.1.0   
       282 slackware 10.0.0 

   The jump in 'SuSE' may represent nothing more than improved detection
   in the 3.2/3.1 scripts vs. 3.0.  We used to have a huge 'unknown' count.


Running under pretty much any *nix is at least theoretically possible.
But it takes interested party/parties and access to resources - some of the
things that ntop does such as libpcap and loading plugins are tied tighter
to the OS than you might like.

There are sections below about each specific OS.


--------------------------------------------------------------------------------
-----Features-------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. What determines the features of ntop?
A. <laugh>Whatever Luca wants</laugh>


Q. Why did you do this "x" instead of feature "y"?
A. Don't know.  I could guess...

   Imagine you are the network manager for a large University network and have
   to crack down on users who are illegally exchanging copyrighted files or
   using University resources to run a business without paying for the resources
   being consumed.

   or

   You are a major vendor of infrastructure, whose customers are using networks
   for new features such as storage area networks and you want to give them
   the ability to monitor these.

   or

   You have a really cool technology that you've just donated to the community
   via an RFC and wish to jump-start adoption of it.

   or

   You are moving heavily into IPv6 and need to be able to monitor your 'new'
   network just like you monitored the IPv4 one.

   or (my favorite)

   You really, really, really hate that ntop generates such lousy html code
   and you decide to scratch that itch.

   or (what should happen)

   A major corporation, with out resources, time, skills and/or inclination
   to do it themselves sponsors the development of a feature that's critical
   to him/her.

   or

   Then again, it could just be because it's cool...


Q. Could ntop do "x"
A. Probably - as long as it doesn't move the tool away from it's purpose and
   it's strengths, almost anything is possible - especially as a plugin.


Q. Will you do "x"
A. Maybe - if it's of interest to a developer, or you provide the code such
   that it can be merged in, or if you're willing to sponsor the development
   effort (contact us through http://www.ntop.org/consultancy.html).


--------------------------------------------------------------------------------
-----Documentation--------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Why isn't there (any)(more)(better) documentation.
A. (A personal peeve from Burton...)

   I get real tired of people complaining that there isn't any documentation
   and then being unwilling to contribute even the simplest stuff.  I've said
   I'll edit and assemble whatever people send me... and since I started working
   with ntop in November 2001, I've received maybe six pages of stuff.

   I'm trying to get people - who aren't coders - to contribute to ntop the
   project.  The contribution that ANYONE can make is "documentation".  A task-
   specific HOWTO... some sample screen shots... An FAQ entry...

   I've tried being nice.
   I've tried asking.
   I've tried shaming people into it.

   What have I gotten?    Zip.

   Nasty is all that's left...  This is your fair warning.  If you show up on
   the ntop mailing lists and complain about documentation, you will get
   blasted.

   -----Burton


Q. Ok, where can I find what does exist.
A. http://www.ntop.org has pointers and some (very out-dated) documents.

   The documentation in the docs/ directory, the Documentation files at SourceForge
   and some at http://www.ntopsupport.com are basically all that there is.

   Search the ntop mailing lists at gmane, http://search.gmane.org. The lists
   are called gmane.linux.ntop.general and gmane.linux.ntop.devel.

   Please contribute to the ntop community by writing things up for inclusion
   in this FAQ or other documents!


Q. I can't find a file at SourceForge!
A. You can reach the archives through any SourceForge mirror, or the main site:

      http://prdownloads.sourceforge.net/n/nt/ntop

   It has ALL the files unless we explicitly delete them...


--------------------------------------------------------------------------------
-----Problems-------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. I have a problem...
A. Make sure it's a supported release!

   We support only the current versions of ntop.  This is either:

       * the last release, e.g. v3.1.
       * the current cvs (cvs.ntop.org)
       * the latest development version posted at SourceForge (if one)

   If you use a port/package and the latest version available for your
   OS is some release candidate from a year ago, sorry.  Contact the
   packager and ask them to get current.

   Luca usually asks people to try the latest cvs (development) version
   because problems are frequently already fixed in there.


Q. I'm running something supported and I've tried the latest & greatest.
   Still, I have a problem.
A. Read the ntop mailing lists.

   The ntop mailing lists are at
      http://listgateway.unipi.it/mailman/listinfo/ntop
      http://listgateway.unipi.it/mailman/listinfo/ntop-dev
   and
      http://listgateway.unipi.it/mailman/listinfo/ntop-misc

   If you're having non-user problems OR you are using the cvs, you should be
   reading and posting ntop-dev (for example, the cvs commit messages are posted
   there).  Stuff unrelated to baseline ntop (PF_RING) belongs in ntop-misc.

   You can read the lists through gmane (or other gateways) if you don't
   want to subscribe, but only subscribers can post.

   You can download the older messages in large chunks from the mailing list
   subscription pages.  Look for "To see the collection of prior postings to
   the list".


Q. I looked and I didn't find my problem.
A. Join the mailing list(s) and ask for help.

   Before you post, check the "HowTo Ask for Help" at the end of this FAQ.

   Please, if at all possible, use the built in PR_ form (the little 'bug' icon
   on the 'About' tab).

   Guidelines for asking questions:

   ONE and only ONE problem / issue / question per message.

   With a meaningful subject.

         The goal is that if you're asking a common question, the
         subject would have allowed you to find it in the back
         traffic for the mailing list.

   Post the information about your environment we ask for.

         We STRONGLY suggest you use the automatically generated "Problem
         Report" form that since it contains much of the necessary information.

   Make sure you're in a supported environment (./configure --showoses).

         If it's an unsupported environment, we're interested in your efforts to
         make ntop work, but we don't have the time, resources, knowledge and/or
         interest to do it ourselves.

   For software 'crashes', please run ntop under the gdb debugger and capture
   the full failure information.

         Brief instructions on using gdb are at the end of this file (docs/FAQ).


Q. I posted to the list and nobody answered me.
A. ntop is open source, and the lists are a community resource.  If nobody
   answered your question, then nobody knew the answers off-hand and nobody
   wanted to spend THEIR time solving YOUR problem.


Q. Do you offer paid support?
A. Yes - contact us through http://www.ntop.org/consultancy.html


--------------------------------------------------------------------------------
-----Configuring ntop-----------------------------------------------------------
--------------------------------------------------------------------------------

Q. What does ./configure do?
A. ./configure checks for the tools installed on your system - configuring
   ntop to compile with the ones you have and skip the ones you don't (or
   to tell you if you're missing something critical).


Q. Why bother - just compile the code.
A. Then you would have to have a machine configured EXACTLY like Luca's.
   Nothing else would work.  Various OSes and Linux distributions package
   the files in different ways and put them in different places.  Plus some
   packages put files into directories with release information in them, etc.


Q. OK, so ./configure
A. Is how you tell ntop where to find things.  A lot of stuff it can figure
   out on it's own, but if things get put in 'strange' places, ntop's
   ./configure has switches you use to tell where to find things.


Q. And the list is?
A. ./configure --help shows the whole list.  It's a bit confusing because
   there are standard options and ntop options mixed in there.

A. So, first let's look at the 'where are things' options.  There are two
   types of files ntop is looking for, '.h' files and libxxxx files.

   .h files are also called 'includes' and libxxxx files are called libraries
   or lib files.

   .h files are the C source for functions provided by the OS or by libraries.
   They are typically in a directory named /usr/include, but they can be
   placed ANYWHERE.

   lib files are compiled libraries of these functions (the .h tells ntop
   how to call something, the lib file is the actual code).  Their names
   usually begin libxxxx (so the library gd is named libgd).  

   By convention, libraries end in .so or .a.  A .so library is a shared
   library (Windows DLL), where one copy of the library is used by all
   programs that want it's functions.  A .a library is a non-shared or
   static library, which must be merged (the technical term is linked)
   with the code.

   ntop uses both - the myrrd library is a static, .a library.  When it
   comes to things like libpcap or libgd, we use shared (.so) libraries.

   Library files are typically placed in /usr/lib, where the gnu linker
   (ld), 'knows' automatically how to find them.  However, from OS to 
   OS and distribution to distribution, there are many other common places.
   Some OSes even have a file telling ld all the places to look.


Q. So ntop looks for these .h and library thingies in a couple of places.
   What if it doesn't find them?
A. If a basic ./configure can't find something, you'll have to tell ntop 
   where to look.

   It's complex and OS/distro dependent. For example, if you install libgd
   from the Sun Freeware site on to a Solaris machine, the files get put
   into /usr/local/include and /usr/local/lib, which are not on the lists
   of 'standard' places for Solaris' versions of gcc (the Gnu c compiler)
   or ld.  So to compile ntop, you have to tell gcc to look in these
   additional locations.

   The things ntop might be looking for are in this part of the
   ./configure --help output:

       +-External-source-locations:-------------------------------------------------+
       --with-pcap-root=DIR        LBNL pcap located in DIR
       --with-pcap-lib=DIR          or libpcap located in DIR
       --with-pcap-include=DIR      or pcap.h located in DIR
       --with-gdbm-root=DIR        gdbm located in DIR
       --with-gdbm-lib=DIR          or libgdbm located in DIR
       --with-gdbm-include=DIR      or gdbm.h located in DIR
       --with-zlib-root=DIR        zlib located in DIR
       --with-zlib-lib=DIR          or libz located in DIR
       --with-zlib-include=DIR      or zlib.h located in DIR
       --with-gd-root=DIR          gd located in DIR
       --with-gd-lib=DIR            or libgd located in DIR
       --with-gd-include=DIR        or gd.h located in DIR
       --with-libpng-root=DIR      libpng located in DIR
       --with-libpng-lib=DIR        or libpng located in DIR
       --with-libpng-include=DIR    or png.h located in DIR
       --with-ossl-root=DIR        openSSL located in DIR
       --with-ossl-lib=DIR          or libssl located in DIR
       --with-ossl-include=DIR      or ssl.h located in DIR
       --with-localedir=DIR        LOCALE files located in DIR (i18n)
       +-----------------------------------------------------------------------+
     
   You can see that there is a pattern, pretty much every --with-xxxxx-root
   has a --with-xxxxx-include and --with-xxxxx-lib option.

   So, if ntop tells you it can't find something, do this - first look for the
   File on your system:

     $ locate pcap.h
     /usr/include/pcap/pcap.h

   (If you don't have locate, this works too:
     $ find / -type f -name "pcap.h"
   )

   And then tell ./configure via --with-pcap-include=/usr/include/pcap

   (Some OSes are smart enough to look in a subdirectory of the standard
   location, but others aren't).

Q. Ok, but why three options?
A. You use either the --with-xxxxx-root option OR either/both of the others
   at a time.  But ntop really only looks at the  --with-xxxxx-include and
   --with-xxxxx-lib options.

   Internally, --with-xxxxx-root=/a/b/c is translated into 
   --with-xxxxx-lib=/a/b/c/lib and --with-xxxxx-include=/a/b/c/include
   (that's the usual pattern).

   Now sometimes libraries are installed logically - if the pcap.h file is in
   /usr/local/pcap/include, the library (libpcap.so) is probably in
   /usr/local/pcap/lib.  Sometimes they are not logical and you will have
   to use the split options.

   The --with-pcap-root=/usr/local/pcap is shorthand for the two options,
   --with-pcap-include=/usr/local/pcap/include and
   --with-pcap-lib=/usr/local/pcap/lib.


Q. Oh Ghu - aren't there any short cuts.
A. For the first time you try ./configure, there's a script on SourceForge in
   The user contributed area that will try to build the ./configure line for
   you.


Q. And the OTHER options
A. There is a set that tells ntop where to install stuff.  For simplicity,
   the two you might want to change are:

       --prefix=PREFIX         install architecture-independent files in PREFIX
                               [/usr/local]
       --datadir=DIR           read-only architecture-independent data
                               [PREFIX/share]


   --prefix tells ntop where to install the various files.  The default value
   is /usr/local, which is where most non-OS software normal goes.

   A common choice for libraries (such as pcap) is --prefix=/usr, which puts
   things like .h files in places easier to automatically find (/usr/include).
   --prefix=/usr certainly works for ntop.  --prefix=/opt is another choice.

   The 'proper' choice turns on which model of file organization the OS and
   sysadmin prefer.  That's a fight I'm staying out of.

   --datadir tells ntop where to put its databases and output files.  The
   default is /usr/share/ntop, but that will give some sysadmin's agita.
   Another popular choice is --datadir=/var, which puts all the files in
   /var/ntop.  That may be attractive especially if you make /var/ntop
   a separate partition, so the rrd files don't eat all your disk space.


Q. What's --enable-iknowbetter        Override WILLFAIL
A. There are some error messages where it's possible that thing work (now)
   that didn't used to, or you're doing development and don't want ntop
   to stop you from doing something, or there's an error message that you
   have skipped before without getting bitten.

   --enable-iknowbetter will print the message but not stop ntop from
   finishing ./configure.

   Use it at your own risk.


Q. What are the --enable and --disable options?
A. These (and the with/without options) pretty much do what you think - they
   enable or disable large chunks of ntop functionality:


       +--ntop-specific:------------------------------------------------------------+
       --enable-sslv3              enable ssl v3 support [default=disabled]
       --enable-sslwatchdog        enable Watchdog for ssl hangups [default=disabled]
       --disable-plugins           disable compilation of plugins [default=enabled]
       --enable-static-plugins     Enable static linked plugins sntop, default=dynamic]
       --enable-ignoresigpipe      Ignore SIGPIPE errors [default=do not ignore]
       --disable-snmp              Disable SNMP support [default=disable]
       --enable-i18n               Enable (limited) internationalization [default=disabled]
       --enable-jumbo-frames       Enable Jumbo (9K) Ethernet frames [default=disabled]
       --disable-ipv6              use IPv6 [default=enabled]
       +-----------------------------------------------------------------------+

       +--external-packages:---------------------------------------------------+
       --without-ssl               disable HTPPS support [default=enabled]
       --without-zlib              disable zlib [default=enabled]
       --with-tcpwrap              enable use of TCP Wrapper [default=disabled]
       +-----------------------------------------------------------------------+


Q. What ELSE?
A. There are some so called 'environment variables' you can set that change
   things too.  The common ones are:
        CC          C compiler command
        CFLAGS      C compiler flags
        LDFLAGS     linker flags, e.g. -L<lib dir> if you have libraries in a
                    nonstandard directory <lib dir>
        CPPFLAGS    C/C++ preprocessor flags, e.g. -I<include dir> if you have
                    headers in a nonstandard directory <include dir>
        CPP         C preprocessor
      
   You would use these variables to override the choices made by ./configure
   or to help it to find libraries and programs with truly nonstandard
   names/locations.

   The best place to look for examples of these environment variables are in
   the OS/distribution files in the configureextra directory.  For example,
   (again picking on Solaris), we use LDFLAGS to tell ld to look in some
   common Solaris locations for libraries via this:

     LDFLAGS="-L/usr/local/lib -R/usr/local/lib ${LDFLAGS}"

   What that does is add /usr/local/lib to the locations that ld will check.


Q. ./configure dies in some strange horrible way complaining about version
   conflicts, lunar phase or whatever.
A. The process of creating portable cross-platform scripts for building
   software is ugly and hard and prone to failure.  There are tools
   (released by gnu) to help, named automake, autoconf and libtool.
   (Collectively the auto* tools).

   These tools take basic files and generate more complex files based
   on a series of rules, conventions and macros (much of which is poorly
   or undocumented).  Other processes (e.g. ./configure) take these files
   and generate more complex files based on a different series of rules,
   conventions and macros.  This ultimately produces the 'Makefile', which
   a program called make uses - based on (...wait for it...) yet another
   series of rules, conventions and macros - to actually compile the ntop
   source.

   There are interdependencies among the tools, partial support for older
   versions in some releases, but not in other releases and many more
   problems.

   Different OSes and different Linux distros ship with wildly different
   versions.  Some even have scripts that attempt to analyze the file and
   pick the correct version (which just means that trying to code a file
   with multiple version support confuses the tool).

   So if you have a basic, total failure of ./configure, it's usually
   the auto* tools.

   ntop used to ship with a manual script that rebuilt the generated files
   according to the version(s) of the tools installed on the system you were
   using to build ntop.  Thus the standard answer was 'run ./autogen.sh -1'.

   But, this meant that you had to have these 'developer' tools installed
   and caused much problems and gnashing of teeth.

   So we spent 100s of hours rebuilding the scripts to be totally independent
   of having the tools installed on your system - only to run into problems
   because the xyz 1.6.1 tool installed by default on OS A isn't quite
   compatible with the 1.6.3 version on OS B.

   So we put technology in place to automatically detect tool versions and
   rebuild the generated files if necessary.  That meant you had to have
   these 'developer' tools installed and caused much problems.

   So we rebuilt the scripts AGAIN and AGAIN, dropped support for old versions
   of the tools and finally reached a point where it works for most reasonably
   current platforms.  This is a compromise:

       Systems that don't have the tools installed usually work.
       Systems that have bleeding edge versions of these tools may break.
       Systems with very old versions also may break.

   The technology to detect versions and rebuild the script files if
   appropriate is still in there, but it's disabled from normal use.

   If you have the auto* tools installed and have ./configure problems, you
   can activate the automatic rebuild feature via:

      $ export NTOPAUTOREBUILD=yes
      $ ./configure ...

   If the rebuild fixes it, that's great.  Regardless, please report the
   problem to the ntop mailing list.  Please don't paraphrase the messages
   cut & paste the ACTUAL MESSAGES into your report.  Also, please let us
   know the version(s) of the tools installed on your system:

      $ aclocal --version
      $ autoheader --version
      $ autoconf --version
      $ automake --version
      $ libtool --version


Q. I can't build ntop... (auto* tools)

A. ntop has been tested with and developed for various versions of 
   the "GNU auto* tools" (automake, autoconf and libtool).

   The basic requirements are

   automake 1.6.1 or higher  (automake 1.4 and 1.5 may or may not work)
   autoconf 2.51 or higher   (autoconf 2.13 will NOT work)
   libtool 1.4 or higher

   But that simple statement hides a lot of complexity and many potential
   problems.  Surprisingly, this isn't solely ntop's fault, it's the 
   combination of GNU auto* tools version to version compatiblity and 
   because we're trying to distribute configure and make files that work
   on all versions.  If ntop were less complex or less clever about 
   trying to build a working program even with many components missing,
   things would work a lot better.

   The big change between autoconf 2.13 and 2.5x generated scripts is that
   the ./configure and make steps are supposed to run the auto* commands
   automatically if they're required.  As we've seen, this doesn't always
   work!  

   If you run into problems, you can ALWAYS recreate the generated files 
   via this procedure:

      rm -f acinclude.m4 aclocal.m4 Makefile.in config.h.in configure Makefile

      find current versions of libtool, config.guess and config.sub and cp
      them into your working directory.

      cat acinclude.m4.ntop libtool.m4.in > acinclude.m4
      aclocal
      autoheader
      autoconf
      automake --gnu --copy --add-missing

   and then:

      ./configure ...
      make
      make install

   as usual.


Q. But how does it all hang together?
A. There's a vsd (Visio)/pdf in the docs directory - ntop-autotools.pdf.


--------------------------------------------------------------------------------
-----Compiling ntop-------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Which packages/libraries do I need to compile ntop:

    Note: In some cases the minimal header files for a tool will be in
          one "package" and the execution library in another. ntop needs
          both so that the ./configure test finds the tool. It's usually
          safest to install both the tool and development packages!

         (Some packages will have additional packages as pre-requisites)

       gdbm (We have people using 1.7.3, 1.8.0, 1.8.2 and 1.8.3)

       libpcap
          We recommend 0.8.3, but ntop works with most 0.7.2 and later versions. 

          STAY AWAY from older versions, especially under Linux.

          Note: Building libpcap requires: bison/flex


       gd 
          http://www.boutell.com/gd/gd.html - note that many distros have both
          the 1.8.x and 2.x series installed for differnt packages that require
          them.  This occasionally causes problems.

       libpng (1.2.x - although ntop works with 1.0.x versions, the two libpng
              versions are not compatible and will often cause version conflicts
              and crashes).

       glibc

       cpp

       gcc most versions of gcc should work, but there are no promisses.

       libtool  1.4+ (distribution is built with 1.4.2)
                     (there are successes reported with 1.3.4 or 1.3.5 -
                      use --enable-iknowbetter)

       gawk/mawk
          Note: If all you have is original awk, then ./configure will not work.

Q. Will the "xyz" compiler work instead of gcc?
A. We explicitly REQUIRE gcc.  

   We know that under many systems, a compiler called cc is available. On some,
   it's a symbolic link to gcc, BUT, when invoked as cc, it often triggers 'old'
   behaviors for cc compatibility.  On others, cc is a retarded compiler just good
   enough for compiling the kernel.

   The one exception is Sun's cc, which - since Luca used to do a lot of development
   on Solaris - was pretty well supported.


Q. The auto* tools?
A. Only if you are going to rebuild the distributed script files:

           autoconf 2.5+ (distribution with 2.57)
           automake 1.6+ (distribution with 1.8.3)


Q. Optional libraries?
A. Lots.  The most common is:

       openssl (for https:// support)
          Note: For security reasons you should only have the most current
                Version of openssl installed.


Q. ntop doesn't compile.
A. Sure, it does - for me :-)


Q. Right smarty-pants...
A. First, check if you're using a supported OS.

   The long and short of it is that ntop works under Linux, Mac OS X and
   FreeBSD.

   Luca distributes a Win32 version through http://shop.ntop.org but charges a
   convenience fee.  Or you can fire up Microsoft's Visual C++ or Visual C++ .Net
   and compile the Win32 version that way.

   It should work under MinGW - see the BUILD file in docs.

   Anything else, you are in untested waters.  Some cases we know there are
   problems, others we just haven't tested.


Q. Ok, it's a supported environment and it still won't compile.
A. Did you run ./configure?  And did it complete successfully?

   Usually 'compile' problems for supported platforms are a missing
   (critical) library or header file, but the user ignored (didn't see)
   the error/warning message and tried running make anyway.

   ./configure checks a lot of things.  When it's looking for
   headers and libraries, ntop will report KEY information and PROBLEMS
   in large, set-off, lines:

     *******************************************************************
     *
     * NOTE: Building ntop for a supported platform
     *       This means we expect ntop to work without major issues
     *
     *            'i686-pc-linux-gnu'
     *
     *    Please keep the ntop-dev mailing list updated with any
     *    successes you have or problems you encounter...
     *
     *   Support for this platform was most recently verified for
     *
     *     RedHat7.2 w/ updates           ntop 2.1.51   on 2002-10-21
     *     Suse i686, 2.4.18-4GB-SMP      ntop 2.1.51   on 2002-10-24
     *
     *******************************************************************

   READ THESE BOXES.  Even if you don't read the rest of the output, read
   the boxes.  ntop can work around a lot of problems (missing libraries)
   by disabling features that need them.  If, for example, you don't have
   zlib, ntop will compile a version that doesn't output compressed html
   pages.  If you don't read the boxes, you will never know.

   READ THE MESSAGE BOXES!

     *******************************************************************
     *                                                                 *
     * NOTICE:  I know you're used to ignoring output from ./configure *
     *                                                                 *
     *          ntop has a lot of complexity and interdependences.     *
     *                                                                 *
     *          Please, please AT LEAST read the stuff in these        *
     *          boxes!                                                 *
     *                                                                 *
     *>>> The ACTION taken is shown prefixed '>>>'                     *
     *                                                                 *
     *    If the ACTION is unacceptable,                               *
     *??? The REMEDIATION plan is shown prefixed with '???'            *
     *                                                                 *
     *******************************************************************

   The box will tell you what's wrong, what ntop did and often how to fix it
   if you don't like ntop's fix.

   READ THE MESSAGE BOXES!

   Hint: It may sometimes be that you're missing the header files (often those
   are in a -devel rpm if you're running RedHat)

   If you see a message box and don't understand why ("I'm sure that
   header file is present"), then look at a file called config.log.  Search
   for the specific header or library reported in the message box and you will
   see the detailed compiler/linker error messages.

   For example, ./configure reports:

        checking for linux/if_pppox.h... no

   The first thing to so is check if it's on your system:

     $ locate linux/if_pppox.h
     /usr/include/linux/if_pppox.h

   (If you don't have locate, this works too:
     $ find / -type f -name "ethertype.h"
   )

   Open up config.log and look for if_pppox.h:

     configure:13086: checking for linux/if_pppox.h
     configure:13103: gcc -c -g -O2 -Wshadow -Wpointer-arith
      -Wmissing-prototypes -Wmissing-declarations -Wnested-externs  -fPIC
      -DLINUX conftest.c >&5
     configure:13158: parse error before '/' token
     In file included from configure:13160:
     /usr/include/linux/if_pppox.h:38: `ETH_ALEN' undeclared here (not in a
         function)
     /usr/include/linux/if_pppox.h:39: `IFNAMSIZ' undeclared here (not in a
         function)
     /usr/include/linux/if_pppox.h:40: confused by earlier errors, bailing out
     configure:13106: $? = 1
     configure: failed program was:
     | #line 13091 "configure"
     | /* confdefs.h.  */
     |
     | #define PACKAGE_NAME "ntop"
     | #define PACKAGE_TARNAME "ntop"
     | #define PACKAGE_VERSION "2.2.91"
     | #define PACKAGE_STRING "ntop 2.2.91"
     | #define PACKAGE_BUGREPORT ""
     | #define PACKAGE "ntop"
     | #define VERSION "2.2.91"
     | #define STDC_HEADERS 1
     ...
     | #define HAVE_NETINET_UDP_H 1
     | /* end confdefs.h.  */
     | net/if.h netinet/if_ether.h
     |
     | #include <linux/if_pppox.h>
     configure:13123: result: no

   You see the actual test program, the actual compile line and the error
   messages.

   Before reporting it to us, chase down where those missing items are declared:

     $ grep -R 'ETH_ALEN' /usr/include/* | grep '#define'
     /usr/include/linux/if_ether.h:#define ETH_ALEN  6
     /usr/include/net/ethernet.h:#define     ETHER_ADDR_LEN  ETH_ALEN

   And post that information with your error report.  The reason is that these
   field definitions are often placed in very different places in different
   OSes and even in different distributions.

   FWIW, if you look in the older configure.in:

      AC_CHECK_HEADERS([linux/if_pppox.h], [], [], [net/if.h
         netinet/if_ether.h])

   should have been

      AC_CHECK_HEADERS([linux/if_pppox.h], [], [], [#include <net/if.h>
      #include <netinet/if_ether.h>])

   because the macro doesn't do the #include automatically.


Q. Nope, it's not ./configure...
A. If it's not the configuration, then it's usually a problem with your specific
   system, either:

        - A new release of a supported OS.
        - An uncommon option/configuration of a supported OS.

   In other words, something is different from what we've seen or expected.

   Review the output from make.  The error message will usually give you a
   Somewhat cryptic description of what's wrong and where.  Look for messages
   about missing files.  Post as much information as you can - do locates for
   the missing files, etc.  The more you give us the less we will have to ask
   you to provide.

   Remember, we can't see your box - all that the people on the list see is the
   information you give in your message.


Q. Compile dies because it's missing depcomp
A. automake/autoconf issue.  The problem should have been fixed. If not, just
   Copy the missing file (or make a symbolic link) into the ntop source
   directory.

   It's in /usr/share/automake on my Linux boxes.  Another user reports it is in
   /usr/local/share/automake in sun8.

   If you have automake installed, this will do it automatically:

       $ automake --add-missing --gnu -c


Q. Make fails with a message about being unable to create a .deps file.
A. Check the permissions on the (hidden) .deps (and .libs) directories - if root
   owns them your non-root userid may not be able to create files in there.


Q. Make fails with a message like this:
     /bin/ld: Warning: size of symbol `pcap_open_dead' changed from 100 to 67
         in pcap.o
     collect2: ld returned 1 exit status
     make[2]: *** [libntop.la] Error 1
     make[2]: Leaving directory `/linuxadmin/ntop/ntop2.2.3/ntop-2.2.3'
     make[1]: *** [all-recursive] Error 1
     make[1]: Leaving directory `/linuxadmin/ntop/ntop2.2.3/ntop-2.2.3'
     make: *** [all] Error 2
A. Ah yes, size of symbol...changed.  It means you have a conflict between 
   the version of the shared library (be it libpcap, libgd, whatever) that
   was used to compile ntop with the version that was used to link ntop.

Q. 'splain please...
A. If you break down a typical link line:

      gcc -shared  address.lo ... ntop_darwin.lo
          -L/usr/local/include
          -L/usr/local/lib
          -L/linuxadmin/ntop/ntop2.2.3/ntop-2.2.3/myrrd
          -lpthread -lresolv -lnsl -lz -lc -lssl
          -lcrypto -lgd -lpng -lpcap /usr/lib/libgdbm.so
          -lmyrrd
          -Wl,-soname -Wl,libntop-2.2.3.so
          -o .libs/libntop-2.2.3.so

   You see the mix of -L and -l parameters.  The -L parameters ADD 
   additional places to look for the shared libraries, which are in 
   addition to the 'standard locations' for the system.  The -l
   parameters tell which libraries to include.

   Read the man pages (man ld, man ld.so, etc.)

   The 'standard locations' are very system dependent, but usually
   include /usr/lib and /lib.  PLUS whatever is (under Linux) in the
   ld.so.conf file, for example

      $ cat /etc/ld.so.conf
      /usr/kerberos/lib
      /usr/X11R6/lib
      /usr/lib/qt-3.0.5/lib
      /usr/local/lib

   So, on this system, to resolve a library, ld looks in the -L values:

      1. /usr/local/include
      2. /usr/local/lib
      3. /linuxadmin/ntop/ntop2.2.3/ntop-2.2.3/myrrd

      And then the 'standard' places:

      4. /usr/kerberos/lib
      5. /usr/X11R6/lib
      6. /usr/lib/qt-3.0.5/lib
      7. /usr/local/lib
      8. /lib
      9. /usr/lib

   Similarly, if you break apart the gcc COMPILE line and scrap the dups,
   you'll have a different set of places where gcc looks for the .h files.
   For those, it's the -I parameters plus whatever is 'standard' on your
   system, which is dependent on the specific gcc port:

   /bin/sh ./libtool --mode=compile gcc -DHAVE_CONFIG_H  -DLINUX
       -I. -I/linuxadmin/ntop/ntop2.2.3/ntop-2.2.3/myrrd 
       -I/usr/local/include
      -g -O2 -fPIC -g -O2 -c
      -Wshadow ... -Wnested-externs
       -o ntop_darwin.lo

   Again, read the gcc man page.

   A big problem is that - unlike .so files, it's not real clear what
   directories ARE searched for .h files, only that there is a set.

   With me so far?  Go back to the message... what it means is that:

      * From one set of locations, at compile time, the size of 
        the parameter list for pcap_open_dead was 100 bytes.
      * From the other set of locations, at link (ld) time, the
        library expects the parameter list to be of size 67.

   Danger, Will Robinson...

   The most likely cause of this problem is when you tell ntop to look at
   one location to resolve either the .h or .so and it finds the other,
   'automatically' in a 'standard' location, but the two are actually from
   incompatible versions.

   Say you have libpcap 0.6.2 installed by the os in /usr/lib and 
   /usr/include, but you also install libpcap 0.7.2 in /usr/local/lib and 
   /usr/local/include.

      $ locate pcap.h
      /usr/include/pcap.h
      /usr/local/include/pcap.h

   and

      $ locate libpcap.so
      /usr/lib/libpcap.so.0
      /usr/lib/libpcap.so
      /usr/lib/libpcap.so.0.6.2
      /usr/lib/libpcap.so.0.6
      /usr/local/lib/libpcap.so.0
      /usr/local/lib/libpcap.so
      /usr/local/lib/libpcap.so.0.7.2
      /usr/local/lib/libpcap.so.0.7

   If all you give ./configure is --with-pcap-header=/usr/local/lib

      * At compile time, it finds the pcap.h in /usr/local/lib (0.7.2)
      * At link time, it finds the libpcap.so in the 'default', 
        /usr/lib/libpcap.so which is actually libpcap.so.0.6.2


   ntop did EXACTLY what you told it to do.  The fact that it makes no
   sense is a problem, so you get the error message.

Q. So the real solution is?
A. Give ntop pointers to consistent sets of header and library files, and maybe
   don't have multiple versions of the same library installed at once.


Q. I'm done compiling and it works/doesn't work, what do I do?
A. make install (You'll typically need to be root for this to work)


Q. I'm seeing a lot of "changing search order for system directory" messages?
A. It's not a problem, just confusing.  This warning, which appears if you 
   redefine the system include directory, causes lots of confusion.  Supposedly,
   according to this http://gcc.gnu.org/ml/gcc-bugs/2002-10/msg01080.html,
   it wasn't fixed in gcc 3.2.0, but should be fixed in gcc 3.2.1.

   According to this: http://gcc.gnu.org/onlinedocs/cpp/Invocation.html,
   it's truly harmless on systems which already have /usr/local/include as
   a standard system library:

      -I dir 
      Add the directory dir to the list of directories to be searched for
      header files. See Search Path. Directories named by -I are searched
      before the standard system include directories. If the directory dir
      is a standard system include directory, the option is ignored to ensure
      that the default search order for system directories and the special
      treatment of system headers are not defeated (see System Headers) . 


--------------------------------------------------------------------------------
-----Other ./configure features-------------------------------------------------
--------------------------------------------------------------------------------

Q. How do I update the Vendor Table (MAC address prefixes)?
A. ntop has (in Makefile), a rule to automatically download the latest vendor
   information table from the IEEE, the oui.txt file ntop reads.

   If you are seeing unknown MAC address prefixes (the 1st three units), try
   the full IEEE table.  To rebuild it:

     # make dnvt

   and then copy the new oui.txt over the one installed by ntop originally.

   Also note that the table changes over time - there are almost 600
   Modifications and/or new assignments between the version shipped with
   ntop 2.0 and the version on the IEEE site in February 2002.


Q. How do I update the passive ethernet fingerprint database?
A. ntop has (in Makefile), a rule to automatically download the latest
   ettercap file from SourceForge:

     # make dnetter


   It will also compress the file and tell you how big the old and new files
   were.


Q. How do I update the IP to Country Code table?
A. Again, ntop has (in Makefile), a rule to automatically download the
   latest data and build the file.

     # make p2c

   Note that this is actually a fairly complex script (utils/p2c) and is
   dependent upon data posted to supposedly well known locations by the various
   internet number authorities (APNIC, RIPE, etc.)

   It's a LOT of data to download so don't do this on a site with a slow
   link to the internet.  Also, read the output carefully and DO NOT apply a
   file if there are messages you don't understand.


Q. But the IP -> CC data is wrong.
A. Yeah.  Most people don't care :-(... it's enough to see various pretty
   flags.

   Sadly, the registries often post files with errors in them.  Some Tier
   1 ISPs post their own data to elaborate on the registry data.  Most ISPs
   don't post data.  Some entries in the files make 'logical' sense, but not
   physical sense.

   If you find another set of data, let us know - the shell script is pretty
   easy to follow to add another data source.

   The user who used to care about this (Mr. Anon E. Mouse) used to send me
   a file of corrections to apply to the file we posted with ntop.

   If such a file exists, it would be posted @ SourceForge in the user
   contributed area.

Q. I have installed ntop for OSX but I cannot start it as I have installed 
   some libraries using fink and ntop cannot find them.
A. The easiest thing to do is to do: ln -s /sw/lib/myfinklibrary /usr/local/lib
   Courtesy of Matthew Scholz <matts@discovery.org>

--------------------------------------------------------------------------------
-----Running - Startup----------------------------------------------------------
--------------------------------------------------------------------------------

Q. ntop won't start - I get this message:
  ** FATAL ERROR** ... open of /var/ntop/addressQueue.db failed:
       File open error
A. It's either permissions (the ntop -u userid doesn't have read/write access
   to that file - common if you're upgrading from 2.2 to 3.0).

   The other cause is that there's still an instance of ntop running. Make
   sure you shutdown ntop before starting it.


Q. What is the function of the 'ntop' script in the build directory - should I
   call it or /usr/local/bin/ntop ?
A. (from the comments in the script):

    # ntop - temporary wrapper script for .libs/ntop
    # Generated by ltmain.sh - GNU libtool 1.4 (1.920 2001/04/24 23:26:18)
    #
    # The ntop program cannot be directly executed until all the libtool
    # libraries that it depends on are installed.
    #
    # This wrapper script should never be moved out of the build directory.
    # If it is, it will not operate correctly.

   It allows you to run ntop out of the build directory before doing a "make
   install" by doing all the necessary linkage magic - such as forcing a relink
   if it didn't succeed originally - to the files in .libs.

   Think of it as simulating make install, but not moving stuff to /usr/local
   or wherever.

   Don't use it - it just causes problems...


Q. Which libraries do I need?
A. To run ntop: glibc, gdbm, libpcap
      For https://, add openssl.
      For other tools and compile options, add the appropriate libraries.


Q. ntop seems to run, but the web server isn't up.
A. Set the password - see docs/1STRUN.TXT


Q. How do you reset Admin password if we lost it?
A. Delete ntop_pw.db and follow the procedure in docs/1STRUN.txt


Q. I'm being prompted to set a password, what do I enter.
A. Anything you want, of 5 or more characters.  If ntop can't find the value
   in it's database, it will prompt you to set on.  If this happens every
   time you start ntop, check the permissions on the ntop_pw.db file.


Q. Characters after the 8th are being ignored in my password.
A. Probably.  ntop uses the standard crypt() function provided by the OS.

   While crypt() is required part of the single unix specification, the 
   details are implementation dependent.  Whatever the limitations of crypt() 
   are, that what ntop's limits are.  

   Unfortunately, there's no way to tell programatically what these limits are, 
   you have to refer to the man page for crypt(3).

   There is a "GNU EXTENSION" to crypt() which implies that changing the 'salt'
   ntop uses would allow longer passwords.  That's set via CONST_CRYPT_SALT in 
   globals-defines.h, but AFAIK this hasn't been tested.


Q. How does the @filename option work e.g. /usr/bin/ntop ... @filename ...
A. The text of 'filename', is copied - ignoring line breaks and
   comments (anything following a #) - into the command line.

   ntop behaves as if all of the text had simply been typed directly 
   on the command line.  Multiple @s are permitted in the command
   line, nesting is not.  @s in the file will cause an error.

   Both are displayed on the info.html report, the "Started as"
   shows the actual command line ntop was given and the 
   "Resolved to" shows what ntop processed.

   Started as /usr/bin/ntop -i eth0,eth1 @/root/ntop_parms -d -L

   with /root/ntop_parms containing:

      -p /usr/share/ntop/protocol.list
      -P /usr/share/ntop
      -w 192.168.42.38:3000
      # -W 192.168.42.38:3001
      -u ntop
      --trace-level 3
      -m 12.239.0.0/16,10.113.0.0/16
      -E
      -K

   becomes:

   Resolved to /usr/bin/ntop -i eth0,eth1 -p /usr/share/ntop/protocol.list
               -P /usr/share/ntop -w 192.168.42.38 -u ntop --trace-level 3
               -m 12.239.0.0/16,10.113.0.0/16 -E -K -d -L

   Remember, most ntop options are "sticky", that is they just set an
   internal flag. Invoking them multiple times doesn't change ntop's
   behavior. However, options that set a value, such as --trace-level,
   will use the LAST value given: --trace-level 2 --trace-level 3 will
   run as --trace-level 3.

   It is recommended that you use FULL pathnames for @filename, since
   ntop may have different effective directories when run in different
   ways.  However, you may wish to use relative pathnames to take
   advantage of the different effective directories (say cron vs.
   command line).  Just know where you're starting.


Q. ntop seems to run but I don't see any traffic.
A. Make sure you aren't running against the loopback (127.0.0.1) interface.
   lo shouldn't see much traffic, only that originating on the host destined
   for it (e.g. ping 127.0.0.1).


Q. ntop is unable to open its database file.  Specifically:
   I have following messages while running ntop

     wait please: ntop is coming up...
     24/Jul/2003 15:15:23 Initializing IP services...
     <snip />
     24/Jul/2003 15:15:23 Initializing GDBM...
     24/Jul/2003 15:15:23  ***FATAL_ERROR*** open of ...name... failed: can't be writer
     24/Jul/2003 15:15:23 Possible solution: please use '-P <directory>'
A. Multiple possible choices...

   * Another ntop already running

   (Most common) You forgot the -P parameter, and ntop is using a default value that
   doesn't exist.  Just as the message says, use the -P parameter to point ntop at 
   the directory you want it to use.

   * Some sort of file system problem (non-existent directory, permissions, etc.)

   1. The directory shown doesn't exist.  Create it.

   2. You many not have read/write rights in that directory.
   This can occur if you run ntop both as root and (as recommended) as a non-root user.

   3. Another instance of ntop may already be running, so it has the file open and locked.


Q. What's inside the .db files and what's their format?
A. The files are stored in GDBM format. You can dump their content using tools like
   this: http://tboudet.free.fr/dumpgdbm/. In principle you should not be
   interested in the file content as they are temporary caches and contain the
   configuration as well as the MD5 hash of the web passwords.

   I posted a trivial dumpgdbm.c to gmane - search for it.

  
Q. ntop starts up with this:
   WARNING: Discarded network 172.20.0.0/16: this is the local network.
A. No worries.  The message means exactly what it says - it's a warning that
   you gave the local network as one of the parameter(s) to -m.  Since the
   local networks are always local, ntop doesn't need to make them pseudo-local.


Q. What are ntop's options?
A. There are a couple of options that appear only if they're not compiled in,
   and a few that depend on various external libraries, e.g. openSSL.

   The best way to see what is actually available is to run ntop with the -h or
   --help options and see.   Also read man ntop.


Q. -4 and -6?
A. The default for the ntop web server is to connect to any address and any family
   of addresses, so if the NIC has both IPv4 and IPv6 addresses it should respond 
   to both.  Use these to restrict the web server.


Q. --trace-level?
A. The settings 0..4 control the amount of information logged or displayed.
   DETAIL (5) adds a [MSGIDnnnn] field and (VERYNOISY) 6 or greater adds the 
   file:line where the message originated.

   -t 5 can be useful with a log watch type package, -t 6 is mostly for debugging.

   -t 5:
      [MSGID8757584] OSFP: scanFingerprintLoop() checked 1, resolved 1

   -t 6:
      [MSGID8757584] [ntop:698] OSFP: scanFingerprintLoop() checked 1, resolved 1

   With 3.2 we've added VERYNOISY and (7) BEYONDNOISY ... they do pretty much what they
   say.  Have LOTS of log space available.


Q. ntop starts up find and then seems to die.
A. Are the ntop threads still there?  Use ps to check, for example:
   # ps -C ntop -m -o "uid,pid,ppid,stat,time,command"

     UID   PID  PPID STAT     TIME COMMAND
     501 18327     1 S    00:00:00 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18330 18327 S    00:00:00 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18331 18330 S    00:00:05 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18332 18330 S    00:00:45 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18333 18330 S    00:00:00 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18334 18330 S    00:00:06 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18335 18330 S    00:00:00 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18336 18330 R    00:00:32 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...
     501 18337 18330 S    00:00:41 /usr/bin/ntop -i eth1,eth2 @/etc/ntop.conf...

   The 'PID' numbers match the THREADMGMT: numbers in the log:

      ntop[18326]:   INIT: Parent process is exiting (this is normal)
           ^^^^^ is the thread that issued the message

      ntop[18327]:   INIT: Bye bye: I'm becoming a daemon...
               (18327 is the 'master' thread created as ntop becomes a daemon)

      ntop[18327]:   THREADMGMT[t105061296]: SFP: Started thread for fingerprinting 
               A message is issued for each thread started.
                                t^^^^^^^^^ is the 'internal' thread #. Ignore it
      ntop[18331]: THREADMGMT[t105061296]: SFP: Fingerprint scan thread starting [18331]
                                                                                  p^^^^^ 
               is the child pid.  Each one issues a message as it starts up

   So:

     UID   PID  PPID STAT     TIME is
     501 18327     1 S    00:00:00 Master
     501 18330 18327 S    00:00:00 (internal used by POSIX)
     501 18331 18330 S    00:00:05 Packet processor
     501 18332 18330 S    00:00:45 Idle Host Scan
     501 18333 18330 S    00:00:00 Address Resolution
     501 18334 18330 S    00:00:06 rrd
     501 18335 18330 S    00:00:00 Web Server
     501 18336 18330 R    00:00:32 eth1
     501 18337 18330 S    00:00:41 eth2

   If you connect to the running ntop via gdb, you can see this:

   $ gdb /usr/bin/ntop 18327
   (gdb) info threads
         ^^^^^^^^^^^^ Shows all the threads and their current state

     9 Thread 114696 (LWP 18337)  0x40255c68 in recvfrom () from
            /lib/i686/libpthread.so.0
     8 Thread 98311 (LWP 18336)  0x40255c68 in recvfrom () from
            /lib/i686/libpthread.so.0
     7 Thread 81926 (LWP 18335)  0x420dcc31 in select () from
            /lib/i686/libc.so.6
     6 Thread 65541 (LWP 18334)  0x420b0226 in nanosleep () from
            /lib/i686/libc.so.6
     5 Thread 49156 (LWP 18333)  0x40252a35 in __pthread_sigsuspend ()
            from /lib/i686/libpthread.so.0
     4 Thread 32771 (LWP 18332)  0x420b0226 in nanosleep () from
            /lib/i686/libc.so.6
     3 Thread 16386 (LWP 18331)  0x40252a35 in __pthread_sigsuspend ()
            from /lib/i686/libpthread.so.0
     2 Thread 32769 (LWP 18330)  0x420db1a7 in poll () from
            /lib/i686/libc.so.6
     1 Thread 16384 (LWP 18327)  0x420b0226 in nanosleep () from
            /lib/i686/libc.so.6

   (gdb) thread 2
         ^^^^^^^^ switch to thread 2 and see it's state

   [Switching to thread 2 (Thread 32769 (LWP 18330))]#0  0x420db1a7 in
            poll () from /lib/i686/libc.so.6

   (gdb) info stack
         ^^^^^^^^^^ check the call stack

   #0  0x420db1a7 in poll () from /lib/i686/libc.so.6
   #1  0x4024f9de in __pthread_manager () from /lib/i686/libpthread.so.0
                     ^^^^^^^^^^^^^^^^^ running __pthread_manager


Q. All the threads seem to be running ok.
A. Check if there's one that looks like this:

     4 Thread 32771 (LWP 27295)  0x420cd207 in sched_yield () from
            /lib/i686/libc.so.6

   Let ntop run a few more seconds (cont command), then check again.

   If it's frozen in the sched_yield, you're probably tripping a deadlock 
   situation that we've seen but don't understand - There's a lot of
   stuff on the 'net, including what seems to be the same problem with
   other resource intensive tools but no clear answers.  

   This happens most often on RedHat Linux 8.0 (9 uses a different
   threading library and the issue hasn't been reported there).  

   Regardless, if you see the deadlock or just have a hung ntop, there is a
   run time option to try: --disable-schedyield.  This option disables the
   calls at a slight penalty to ntop's interactivness.  If you're experiencing
   the deadlocks, try it.

   And, especially if you're running something other than RedHat 8, please
   let the ntop-dev list know about it.


Q. Can I set the admin password from a script?
A. Yes, you can call ntop with the option:
      ntop --set-admin-password=password        


Q. What's this warning about ownership?
A. The first time you run make install and ntop creates a new
   directory for ntop's files, there's a problem.  So you'll
   see this warning box:

      ************************************************************
      ************************************************************

        WARNING: This install created a directory for the ntop
                 files and databases:

                   some directory...

                 This directory MUST be owned by the user
                 which you are going to use to run ntop.

                 The command you must issue is something like:

                 chown -R ntop.ntop <directory>
           or    chown -R ntop:users <directory>

                 man chown to check the syntax for YOUR system

      ************************************************************
      ************************************************************

   Since root created the directory, root is the owner.  And since we don't
   know - yet - what user ntop is going to be run as, we can't issue the
   chown (CHange OWNership) command for you.

Q. And if I don't do this?
A. ntop won't run.  Typically you get a message about a bad -P file or
   endless prompts for the admin password.


Q. I can't merge interfaces (-M option)?
A. Check your plugins and see if either netFlow or sFlow is active.
   Regardless of whether you're using them, if they're active, they
   (silently) force the -M switch on.

   If you have used the web configuration to set this, you may be
   unintentionally overriding your command line.

   With v3.2 there are log messages telling you what sets interface
   merge.

        NOTE: Interface merge disabled by default
        NOTE: Interface merge enabled by default
        NOTE: Interface merge disabled from prefs file
        NOTE: Interface merge enabled from prefs file
        NOTE: Interface merge disabled due to command line switch


Q. Explain -u.
A. -u root means that you are running ntop as root.  It means you don't drop
   root's superuser privledges, so pretty much you can read/write any file.
   It's not a good idea.  While we're not aware of any security problems with
   ntop, programs that run as root are targets.

   -u ntop (or -u whatever) means you run as a normal user and can be assigned
   only the privledges necessary to run ntop.  It also means you can only read/
   write files as ntop.  So less is exposed.

   For even better security, the -u userid should not have a valid shell, so
   people can't use it to login.


Q. What are the options that reduce ntop's workload?
A. These options turn off certain processing (and thus limit ntop's
   functionality), and may be appropriate for high volume installations.

   -b | --disable-decoders

     This flag disables protocol decoders. Use it for better performance
     or if you feel ntop has problems handling these protocols in your
     environment.

     This switch disables code in a number of places throughout ntop, code
     which analyzes specific protocols, but can place additional load on the
     host.  This switch could be used to run ntop on low-end CPUs or where
     ntop is acting as a collector (netFlow or sFlow) and the GUI is not
     required.

     Disabled is the analysis of:

        DNS Sniffing - where ntop captures DNS information from other hosts'
                       requests to reduce the # of DNS requests ntop must -
                       itself - make.

        NetBIOS   \
        NetWare    \
        AppleTalk   -- resource intensive protocol analysis of less
        bootp/dhcp /   common protocols.
        OSI       /

        http (80) - Request success/failure counting on port 80 and other
                    analysis, including "Virtual Host".

        ftp passive session tracking.

        "Wrong Port" monitoring for: http, ftp and smtp (used with the
              -q | --create-suspicious-packets option to dump "suspicious"
              packets to an analysis file)  With this option, ntop checks
              the payload for each new connection, looking for text usually
              present in http, ftp or smtp requests.  If these are not on the
              "normal" ports (http's 80, ntop's 3000 or squid's 3128, ftp's
              21 or smtp's 25) (or there is a non-ftp or smtp request on the
              standard ports), the packet is logged.


   -g | --track-local-hosts

     Use this flag to tell ntop that you care only about local hosts (use
     -m | -- local-subnets to specify local nets).  This flag is useful when
     ntop sees many hosts (e.g. border gateway) but only the local ones need
     to be tracked.

     This switch disables code in a number of places throughout ntop, code
     which allows ntop to track "foreign" hosts (that is ones not local
     according to the IP address(es) of ntop's interfaces or set pseudo-local
     by -m | -- local-subnets).

     Basically, ntop doesn't bother to do DNS resolution on these addresses
     and, for purposes of various counts, uses the "other" bucket instead of
     creating a unique hash table bucket for the specific host.

     This switch could be used to run ntop on low-end CPUs or where ntop is
     acting as a collector (netFlow or sFlow) and the GUI is not required.


   -o | --no-mac

     Specifies that ntop should not trust MAC addresses but just IP addresses.
     This option is useful whenever ntop is started on an interface where MAC
     (Media Access Controller - the low-level Ethernet address) addresses can
     not really be trusted (e.g. port/VLAN mirror in Switched Ethernet
     environments).

     Certain processing is performed differently:

          Hash search is via IP not MAC

     Certain capabilities are disabled:

          Analysis of bootp/dhcp requests
          localRoutersList.html report
          Wrong net mask log message and flag
          Analysis of non-tcp/udp protocols like NetWare and Spanning Tree
          Router listing on Host Detailed report.
          Traffic Matrix report

     (Note that this list is subject to change as we learn more about protocols
      that do/do not depend on the MAC address)

     See also -z | --disable-sessions


   -z | --disable-sessions

     This flag disables tcp session tracking. Use it for better performance or
     when you don't really need the tracking of sessions.

     Also, in situations where the MAC addresses cannot be trusted, ntop may
     - or may not - be able to accurately track tcp sessions.  There is no easy
     way to tell, so this switch puts control back into the users' hands.

     In versions after 2.0 up to & including 2.1.2, the -j | 
     --border-sniffer-mode flag (predecessor of -o | --no-mac) always turned
     this off.  Many users wanted to try turning session tracking back on, and
     did via code patches with mixed results.

     Suggested usage:  If you enable -o | --no-mac, try running ntop with
     sessions enabled.  If the data looks reasonable, congratulations - your
     network allows session tracking.  If the data does not look reasonable,
     then you will also need to disable session tracking with this switch.


Q. -s | --no-promiscuous doesn't work
A. It should work - it's passed to pcap_open_live.

   But, whether the flag is supported by the OS and whether it is actually 
   respected by the interface depends on libpcap.  If it fails, you'll see 
   a message and ntop will refuse to startup.

   Understand that if it actually does work, it means that ntop sees a lot 
   less comprehensive view of the traffic.

   But from the ntop web server, you won't see anything different.

   You can check (*nix) by executing ifconfig on the interface.

   Note that the parameter specifies if the interface is to be put into 
   promiscuous mode because of ntop.

   Even if this parameter is false, the interface could well be in promiscuous
   mode for some other reason.  And in some situations, the interface may be 
   forced by the low level driver into promiscuous mode, without reporting
   this to the kernel!


Q. How does -m | --local-subnets work?
A. This  flag  allows users to specify the subnets whose traffic is considered
   local (called "pseudoLocal" internally).  

   The format is
        <network address>/<# subnet mask bits>[,<network address>/<#  subnet 
         mask bits>

   For instance "131.114.21.0/24,10.0.0.0/255.0.0.0".


Q. (followup) but what does it MEAN?
A. Surprisingly, it means EXACTLY what it says.  Treat traffic on the listed
   subnet(s) as local.

   -m relates to the traffic you see on the wire at the TCP/IP level.
   -m tells ntop something it can't determine by itself.  And that is to 
   treat a range of addresses EXACTLY like it was local.

   For example, on my Cable Modem, I see broadcasts for a number of subnets
   that AT&T (now Comcast) has assigned to this area (I don't see the 
   traffic just the broadcasts).

   If you have VLANs or simply network overlays (two or more networks on the
   same wire, but with separate address spaces).  etc.

   Those are the cases where you use -m.  To tell ntop to treat that traffic
   as local.

Q. Why 
A  ntop differentiates between local traffic and remote traffic.  There are
   actually four classes (although only three are routinely reported) L->L L->R
   R->L and R->R.  Some additional processing is done and some additional 
   reporting is available for L traffic.

Q. I'm confused.  Explain, please!
A. Suppose your IP is 1.2.3.4 with a 255.255.255.0 netmask (a/k/a 1.2.3.4/24)

   Under the TCP/IP protocol, traffic with any address 1.2.3.1 -> 1.2.3.254 does
   not get routed.  It's "local".

   Your buddy is at 1.2.3.9 and the router is 1.2.3.1, so your network looks
   like this:

   the       +--------+
   world-----+ Router +--1.2.3.1--------------------------------------
   +dog      +--------+                | 1.2.3.4             | 1.2.3.9
                                  +--------+            +--------+
                                  |  You   |            |  Buddy |
                                  +--------+            +--------+

   Say you send a packet to your buddy at 1.2.3.9. You build a packet with
   SRC=1.2.3.4 DST=1.2.3.9 and your data and cast it out the wire.   (For
   purposes of this illustration, ignore the fact the your TCP stack would
   recognize the "local" nature of the packet and actually use another, lower
   level protocol, called Ethernet to deliver it.)

   The router (1.2.3.1) looks at it, does the math and ignores it - it's local
   Your buddy (1.2.3.9) looks at it, says - gee, that's me and reads it

   This is L->L traffic.

   Now you send a packet to ntop.org at 131.114.21.9.  Again, SRC=1.2.3.4 and
   Now DST=131.114.21.9.

   The router (1.2.3.1) looks at it, does the math and says - oops, I have to
   send it out to the world. Your buddy (1.2.3.9) looks at it, says - gee that's
   NOT me and ignores it

   This is L->R traffic.

   Now it's perfectly possible to have multiple (physical) networks on the same
   physical wire.  Say that your ISP chooses to put 1.2.4.1-1.2.4.254
   (1.2.4.0/24) on the same wire. (Why would they do this? - maybe it's a big
   pipe and only a few users or whatever).

   A packet from 1.2.4.4 -> 1.2.4.9 is seen by

   The router - no, that too is local, ignore it
   You (1.2.3.4): (1.2.4.9) - not me - ignore it
   Buddy (1.2.3.9) - um... 1.2.4.9 - not me - ignore it

   And that's perfectly legal.

   But what if you are the ISP and you want ntop to see ALL the traffic on that
   wire? ntop will figure out from it's own IP address that the 1.2.3.0/24
   traffic is local, but it will classify the 1.2.4.0/24 as REMOTE.

   And that is what the --local-subnets switch does.  It tells ntop to treat
   that 1.2.4.0/24 traffic as local.

   If there isn't any other traffic on the wire, then telling ntop to treat it
   as local won't change a thing.

   You can always use a packet sniffer, such as tcpdump to scan the traffic on
   the wire and see what's really there...


Q. And internally?
A. ntop is designed as a hybrid packet analyzer, not a pure Ethernet analyzer
   (layer 2) nor a pure TCP/IP analyzer (layer 3).  Most of ntop's displayed
   counts are at the TCP/IP level, and that's what confuses people.  Internally,
   ntop works both at the level of the Ethernet frame and the TCP/IP packet.

   A single MAC address can be associated with multiple TCP/IP addresses.  The
   MAC address -- unless something is horribly wrong on the network or with the
   hardware or somebody is deliberately spoofing it -- is guaranteed to be
   unique and refers to a physical host or network interface.  For many reports,
   ntop displays the information using the MAC address to separate physical
   devices.  Other data is accumulated and displayed at the TCP/IP (level 3)
   layer.

   -m relates to the traffic you see on the wire at the TCP/IP level.
   -m tells ntop something it can't determine by itself.  And that is to treat
   that range of addresses EXACTLY like it was local.

   For example, on my Cable Modem, I see broadcasts for a number of subnets
   that AT&T has assigned to this area (I don't see the traffic, but you get
   the picture) in an overlay structure (two or more networks on the
   same wire, but with separate address spaces).


Q. What about multicast traffic?
A. Multicast traffic uses the 224.0.0.0/4 address range (that is all addresses
   between 224.0.0.0 and 239.255.255.255 - see RFC 1112).  By default then,
   all multicast traffic is treated as 'Remote' by ntop.  

   Your actual network topology may be such that multicast traffic could
   all be local, all be remote or a mixture.  You can use the -m | 
   --local-subnets parameter to force some (or all) multicast groups
   to be counted as 'Local' traffic.

Q. What about VLANs?
A. VLAN traffic is also a special case.  ntop treats all traffic within your 
   machine's netmask definition as 'Local'.  For instance, you may have a /16
   network (netmask of 255.255.0.0), and you are using VLANs to distribute
   the traffic.  In this case ntop will see the a.b.c.d/16 address and classify
   all traffic within this /16 network as 'Local'.

   Don't design your network that way.

Q. But they did!
A. This MIGHT work:

   Use an un-numbered interface, so that ntop doesn't assign any implicit
   'Local' addresses.  Then use the -m | --local-subnets parameter to 
   define what is truly 'Local'.  As long as there aren't too many
   'Local' networks, this should work.  I run something sort of like this
   (but using /24s not subdividing a /16) for my development network.


Q. What are the default protocols ntop monitors?
A. (These are the ones ntop monitors if the user does not supply a -p parameter)
   Check addDefaultProtocols() in ntop.c around line 525.
   The current list (December 2004) is

     Protocol   Ports
     --------   -----

     FTP        ftp ftp-data
     HTTP       http www https 3128      /* 3128 is HTTP cache */
     DNS        name domain
     Telnet     telnet login
     NBios-IP   netbios-ns netbios-dgm netbios-ssn
     Mail       pop-2 pop-3 pop3 kpop smtp imap imap2
     DHCP/BOOTP 67-68
     SNMP       snmp snmp-trap
     NNTP       nntp
     NFS/AFS    mount pcnfs bwnfs nfsd nfsd-status 7000-7009
     X11        6000-6010
     SSH        22
     Gnutella   6346 6347 6348
     Morpheus   1214
     WinMX      6699 7730
     DirectConnect 
     eDonkey    4661-4665
     BitTorrent 6881-6999 6969
     Messenger  1863 5000 5001 5190-5193

   Note that the names come from /etc/services (or your system's equivalent).
   If you add protocols to /etc/services, you can refer to them by name on the
   -p parameter.

   REMEMBER: You must define the list using the format illustrated in the ntop
   man page.  Don't try to read /etc/services.  It will fail.

   The list changes over time as P2P protocols appear and disappear.  Check the
   cvs and diff ntop.c (around line 550 in void addDefaultProtocols() if you
   want the history.


Q. What about protocol XYZZY?
A. The analysis of protocols is very limited and unsophisticated.  But,
   theoretically, if it's there in plain text, we could report on it.
   The more work you can do up front in identifying the protocol (e.g. port #s,
   header structure, etc.), the easier it would be to add.


Q. I am using a /16 (/25 or whatever) mask and I get this message:
      Truncated network size to 1024 hosts (real netmask 255.255.255.0)
A. Yes.  ntop limits each network to 1024 hosts (a /24). If you need more, alter
   the #define for MAX_SUBNET_HOSTS in globals-defines.h and recompile.  Space
   has to be reserved for this many hosts for each network, so the limit exists
   to keep memory usage from growing to absurd levels on people with "class A"
   (/8) interfaces (e.g. 10. or Cable Modems, etc.).


Q. What's with the version check?
Q. Why is ntop connecting to jake.ntop.org?
A. jake.ntop.org is the cannonical name for version.ntop.org, which hosts the xml
   file used for the version check.

   See the man page, 1st run log messages and/or privacyNotice.html page (Click on the
   "this" in this line: "For information on ntop and information privacy, see this page."
   on the 1st page ntop presents).

--------------------------------------------------------------------------------
-----Running - On-going---------------------------------------------------------
--------------------------------------------------------------------------------

Q. Explain L and R - why doesn't ntop double count?
A. Classification is all based on what ntop SEES in the packets.  ntop sees 
   packets and ONLY packets.  Packets have a FROM and a TO address.  Which 
   packets ntop sees is determined by the interfaces it is monitoring.

   Traffic (packets) is classified based on the joint classification of the
   FROM address (L or R) and the TO address (L or R).

   Only in L->L traffic will ntop see sent=rcvd and 'double count'.

                            Host: 192.168.1.x  www.yahoo.com
                                  L->R  R->L   L->R  R->L
                                  S  R  S  R   S  R  S  R
      192.168.1.x>www.yahoo.com
        HTTP GET ...             30  .  .  .   .  .  . 30

      www.yahoo.com>192.168.1.x
        HTTP 200                  .  .  .  8   8  .  .  .
     
      www.yahoo.com>192.168.1.x   .  .  .200 200  .  .  .
        <html>...</html>

      etc.


Q. Why does ntop use so much memory ?
A. ntop holds a lot of information about each host it has seen in an in-memory
   table. Periodically, it looks at all the entries in the table and flushes any
   which have been idle for a period of time.

   You can change the sizing of the table and the flushing interval via #define
   statements in globals-defines.h.

   But realistically, ntop needs enough memory to hold information about what's
   active on YOUR network.

   To reduce memory, monitor fewer protocols or use the filter (-B "bpf filter")
   option to monitor only parts of the network.

   There have been a couple of discussions on the ntop mailing list about ntop's
   memory usage - you might read them (search on gmane).

Q. Database.
A. No way.  This data just can't be stored in a database - while each record (row)
   is small, we're talking 1000s of updates/second.  As in Enterprise Oracle class
   speeds...  

A. ntop's memory usage for host tables depends on the # of hosts it sees in
   packet traffic.  This is NOT, repeat NOT controllable by ntop in ANY way.
   If a user kicks off a port scan, 100s of hosts appear.  If somebody does a
   DOS attack against you, 1000s of hosts 'appear'.  If a user searches Kazza
   for an obscure song, it can probe 4K hosts. etc.

   Lots of those hosts appear, have a few bytes of traffic and then disappear.
   Each host has a variable amount of memory - there's a base structure, some
   optional counter structures and a large # of pointer fields, which may or
   may not be valued for any given host.  It depends on the # of active
   sessions and a lot of other things, but 8-20K is a good guess - I usually
   use 12K as an guestimating size.  Similarly, sessions may appear and
   disappear (http: opens a lot, does a small retrieval and closes them), ssh
   may last for days.  etc.   Memory is consumed tracking them too.

   It's not just one table entry per host - there are a lot of ancillary tables 
   which only get allocated if you have data for them.

Q. Got a guess?
A. Sure.  Look in textinfo.html and you will see the line:

     (very) Approximate memory per host.....4.4KB

   I'm not sure how much I trust this and I wrote it.  This calculation just
   computes the (current usage - baseline) / # of hosts.


Q. So ntop's memory usage is dependent upon?
Q. So what does the purge do?
A. ntop purges idle hosts.  Period.

   Idle being defined as having not had packet traffic for a #define able
   period, 5 minutes by default.

   Because of the amount of linked data, these purges take time (lots of 
   free() calls on all those char* values), so we don't purge in 'real time'.
   First we build a list of idle hosts, then we purge from that list.
   Any idle host is eligible for purge (unless you tell ntop not to purge idle
   hosts, which is usable only on small networks).

   Over time, all idle hosts are purged.  Only to be perturbed by the 
   next burst of activity - say a port scan or everybody logging in back
   in after lunch.  Eventually, there's some form of steady state, but it's
   HIGHLY dependent upon network activity.  Which, remember, is external to 
   ntop and can't be predicted.


Q. Why?
A. Think about the overhead of sorting this huge structure by 'last packet
   seen time'.  1GB of ram is something like 80K+ hosts and takes a long
   time to sort (let alone free).   During which time, on that busy network,
   a couple of dozen packets are processed...  Meaning maybe your list of idle
   hosts is wrong.


Q. So how about a hard memory limit?
A. There's no way to make a hard limit without purging ACTIVE hosts (or
   non-IDLE given ntop's definition of idle).

   Think about it ... you're at that 1GB limit and you find a new host.
   Do you kick off an interim purge (with it's huge overhead?)  And wait
   2 or 3 s for an available slot??  While 1000s of other packets appear??


Q. Soft limit?
A. It might, might, be possible as a soft limit - but it's got a lot of
   issues.

   First off, tracking memory usage of the hosts tables is itself a huge
   job. There are multiple places were stuff is purged, added to the 
   structures as pointers, etc. and there's a queue of purged entries 
   for reuse to cut down on the malloc() calls.

   Secondly, the purge is resource intensive, and has been the cause of
   deadlocks before - you don't dare lock the structures for too long - 
   packets keep arriving, and FAST on the busy network that has the memory
   issues in the first place.  Since you can't lock for long, you can only
   purge a small # of entries.


Q. But what about -x and -X.
A. Ah, yes, grasshopper - you have been reading the man page.  Good user.
   The -x and -X options are pretty crude.  They say "if you have allocated
   n (hosts|sessions), fail the allocation of (host|session) n+1".

   It works in the context of preventing memory exhaustion - if you can calculate
   the right values based on your available memory (and remember that 4K or 12K or
   whatever per host value is an AVERAGE).
 
   And, no, you can't make them memory dependent - it's all ultimately virtual and
   ntop doesn't get indication of whether a malloc() call causes swapping.  It either
   works because the allocation (total) is under your limit (or the hard limit of real
   memory + swap space - OS usage), or it fails.

   This type of limit can't make sure you save only the RIGHT hosts.

Q. These options aren't very smart, are they?
A. No.  They don't know anything about the hosts.  Just the # of them.  So failing
   an allocation might hurt in that the n+1th host might be an important one and yet 
   host #2 is basically trivial junk (the printer that sends and "I'm here" message
   every three minutes).

   It also causes overhead, because the next packet for host n+1 will attempt to
   allocate a record and fail.  Again and again.

   But these options may keep ntop from crashing and that's a good thing.

   If you use them, I would recommend setting them very high - just at the limit 
   of what your memory permits.  Use this as a last resort - say you get hit by
   a worm or virus.

   I recommend that you use other options such as filter and track-local-hosts for 
   the day-to-day controls on what ntop tracks.


Q. Why does ntop drop packets?
A. We used to have an extensive discussion here.  But with Luca's merge of ntop-ht
   into baseline, we shouldn't be dropping packets beyond whatever the OS is dropping.

   Until we've learned enought about ntop 3.2 and following, the whole discussion
   has been moved to FAQarchive.


Q. ntop stops capturing packets, except ARP and other broadcasts.  Why?
A. Check if you have a daemon running that periodically checks for and
   resets interfaces in promiscuous mode?  If that happens, all you
   would see were broadcast packets like ARPs...

   Check back in the log and see if there is a message about the interface
   changing status.  Determine why.


Q. How much horsepower do I need to run ntop on a network of size x?
A. Nobody really knows.  ntop needs enough memory to store the active
   hosts and enough cpu to keep up with the average packet flow.  The
   buffer will handle the occasional peak, but if you see frequent
   lost packets, you're in trouble.

   Note that a few packets occasionally lost isn't a big deal for most
   users.  After all, the network itself has losses - I've seen my AT&T
   Broadband connection have spurts of 30% packet loss.  Ideally in a
   LAN environment, the packet loss should be down in the small #s...
   the Ethernet standard allows 1  error in 100,000,000(10^8), but most
   vendors beat that by a long margin (even as high as 1 in 10^12).

   Of course, those are lab measurements.  In the real world?  Not that good.
   Electrical noise can be a real bugaboo. Remember, at a certain point, if
   the NIC doesn't understand what it's seeing, it throws it away and
   declares an error.  The key is to keep up with the traffic.

   Similarly, the OS kernel does the same thing in it's interrupt handling
   (throw away packets).  Last resort, but better than hanging up the whole
   machine.

   ntop drops packets when the queue gets longer than the permitted length.
   You can see this in the configuration page as # Queued Pkts to Process
   and # Max Queued Pkts.

   One or two or a small number (you pick your tolerance) is ok, but constant
   losses isn't.  What I'm saying is that as long as ntop can keep up with the
   NIC, then the data is as good as it gets...  if ntop can't keep up, then the
   data isn't very good.

   If you have measurements - network size, traffic flow and %CPU used (with
   the hardware info, of course), shoot them over to us on ntop and someday
   maybe we'll be able to give better #s.


Q. What about my Frobozz Model xx Magic Network card? Is it good enough?
A. Probably.  Well, a lot of the cheapos just don't have the buffering 
   and cpu offload of a 3c905c or such.  If the network isn't that busy,
   anything will do.  For a busy network, buy a decent PCI NIC.


Q. Gigabit Ethernet?
A. No clue.  Remember that you can suck a lot more traffic over that
   network than an old PC can handle (i.e. the bandwidth limitations
   of 32bit PCI and PC100/133 RAM, heck even PC2700 DDR).  

Q. Do "zero copy" drivers help?
A. Yeah.  Maybe.  Once upon a time, I read about "zero copy" - look here
   http://people.freebsd.org/~ken/zero_copy/ for the FreeBSD stuff. Quoting:

      "What is "zero copy"?

      Zero copy is a misnomer, or an accurate description, depending on how
      you look at things.

      In the normal case, with network I/O, buffers are copied from the user
      process into the kernel on the send side, and from the kernel into 
      the user process on the receiving side.

      That is the copy that is being eliminated in this case. The DMA or 
      copy from the kernel into the NIC, or from the NIC into the kernel is 
      not the copy that is being eliminated. In fact you can't eliminate 
      that copy without taking packet processing out of the kernel altogether.
      (i.e. the kernel has to see the packet headers in order to determine 
      what to do with the payload)

      Memory copies from userland into the kernel are one of the 
      largest bottlenecks in network performance on a BSD system, so 
      eliminating them can greatly increase network throughput, and 
      decrease system load when CPU or memory bandwidth isn't the 
      limiting factor."

   What's that mean?  It means that BEST CASE, you have to copy the data 
   NIC->Kernel and without zero copy, it happens TWICE.  Then, once you're
   in the kernel, it has to hand the data off to libpcap (another copy) 
   and from libpcap to ntop.  So we're moving the data 3 or 4 times... 
   best case!

   Let's do some off the cuff math.  Looking here
      http://www6.tomshardware.com/mainboard/20020501/ddr400vsrambus-05.html
   shows a table of memory types and max bandwidth (picking a few):

      Label  Name  Effective Clock  Data Bus Bandwidth
      PC100  SDRAM       100 MHz     64 Bit  0,8 GB/s
      PC133  SDRAM       133 MHz     64 Bit  1,06 GBb/s
      PC2700 DDR333      166 MHz     64 Bit  2,7 GB/s

   See the problem?  A fully loaded GigE link is 1 Gb/s.  4x that is 0.5 
   GB/s - so if ALL that's going on is the capture of packets from the network,
   the CPU can keep up.  Maybe.  Those bandwidth numbers are theoretical, best
   case (nice sequential access).  Throw in some other system activity, cache 
   misses and CAS/RAS issues... and um...

   The moral is that if you're going to use ntop to monitor big fat links,
   you need screaming fast iron.


Q. Can I disable logging? Totally?
A. No.


Q. I'm seeing weird "hosts" on my network with names like
   "Bridge Sp. Tree/OSI Route".
   What are they?
A. There is a list of "special" MAC address prefixes in vendor.c,
   specialMacInfo[].  There are blocks of MAC addresses reserved (sometimes not
   formally) for special uses, such as sharing information about Spanning Tree
   for bridges.  These do not have an IP address - they operate at a lower
   level - so nothing gets displayed in some of ntop's fields.

   A reference about protocols at the wire level is here:
      http://www.oreillynet.com/pub/a/network/2001/03/02/net_2nd_lang.html

   If you only want to see TCP/IP, then I suggest you use -B "ip" to filter
   only TCP/IP protocol on your ntop line...


Q. How do I see fully qualified names for all my hosts? Some are netbios
   names!
A. ntop doesn't SEND NetBIOS queries, it sniffs them off the traffic already on
   the network.

   There is only ONE case where ntop uses the NetBIOS names, which is if
   it can't resolve them via DNS (both it's own queries and from sniffing
   responses to other's queries off the network).

   So, if you have a properly functioning DNS, you'll see DNS names.  If
   these are (for example) internal names, unknown to the DNS server, you'll
   see NetBIOS names if they are available.  Lastly, you'll get IP addresses...

   If you do have a DNS, and the name is resolved as part of the default
   domain, you won't see a fully qualified name back from the DNS, so ntop
   won't have that information.

   So, on a real network you'll often get a mix of name resolution types:

    Host                            IP Address      MAC Address
                                                                   Other Name(s)
    netnews.attbi.com               63.240.76.16
    tigger.homeportal.2wire.net     192.168.0.xx   00:D0:09:xx:xx:xx
    homeportal.homeportal.2wire.net 192.168.0.1    00:D0:9E:xx:xx:xx
    swallowtail                     192.168.0.XX   00:A0:CC:xx:xx:xx SWTL [DMN]
    12-xxx-xxx-xxx.client.attbi.com 12.xxx.xxx.xxx 00:D0:9E:xx:xx:xx
    12-xxx-xxx-yyy.client.attbi.com 12.xxx.xxx.yyy


Q. What does this log message (and others like it) mean?
   **WARNING** releaseMutex() call with an UN-LOCKED mutex [hash.c:559] last
               unlock [pid 22176, pbuf.c:2598]
A. Those messages are part of an error check in our mutex handling routines.
   
   If things are working properly, then these messages tell us things are 
   really, really, really wrong.

   releaseMutex() call with an UN-LOCKED mutex means that the corresponding
   accessMutex() call never occured.

   releaseMutex() call with an UN-INITIALIZED mutex means that the 
   access/release occured before the createMutex().

   The file/line comments, e.g. [hash:559], tell you where the access/release 
   was called from (e.g. hash.c @ 559) and where the last recorded release was.

   There are others, look in leaks.c for the entire set.


Q. How serious are they?
A. None of these stop ntop from processing, but they're indications of
   unprotected accesses to shared data areas, which could lead to lost counts.

   In the development cycle after 3.1 was released, I was able to track down
   and fix the remaining cause of spurious messages (but I've thought this was
   fixed before).

   For most users, these warning messages shouldn't be ignored.


   The root of the problem is that POSIX mutexes are stupid and don't record
   a lot of state information.  To debug things like deadlock, it's really,
   really, nice to know where the mutex was locked from.

   So ntop has added 'extra' information to the mutex data for recording this.
   This added data is not protected by the mutex and could get out of sync.
   So we lock a state change mutex whenever we are changing the state data.

   BUT: You can't hold the state change mutex while waiting for the main mutex
   or you deadlock.

   So the 'access' call becomes this pattern (see utils.c):

   lock(statechangemutex)
     set state data

     tryLock(mutex)
     if(fails) {
       unlock(statechangemutex)
       lock(mutex)  <-- which may block, but does not hold the statechangemutex!
       lock(statechangemutex)
     }
     set state data
     unlock(statechangemutex)


Q. Threads. ugh...
A. There's a good set of articles on POSIX threads at
      http://www-106.ibm.com/developerworks/library/l-posix1/,
      http://www-106.ibm.com/developerworks/library/l-posix2/ and
      http://www-106.ibm.com/developerworks/library/l-posix3/.


Q. What does the message "URL security(1): ERROR: Found percent in 
   URL...DANGER... rejecting request" mean?
A. It means that ntop received a request with a percent sign (%) in it, often
   used as part of Unicode exploits against various web servers.  Since there is
   no situation where ntop should process this, we reject it.
   URLsecurity in http.c is the place where these tests occur.


Q. What does the message "Rejected request from address x.y.z.t (it previously
   sent ntop a bad request)" mean?
A. Once you send ntop a request that URLsecurity rejects, the sending address
   goes into a ring buffer on a 5 minute timeout where we simply drop subsequent
   requests...  rather than waste cycles ignoring an attack...


Q. What are the other URL security(#) codes?
A. 1.  Found a % in the request (Unicode problems)
   2.  Found a parameter type code (//, &&, ??)
   3.  Found a directory transversal code (..)
   4.  Found a prohibited (RFC1945) character
   5.  Found a bad extension


Q. ntop doesn't report any traffic at all.
A. Understand how ntop works:  It simply listens on the interface(s) for
   packets, then counts and interprets them.  If there aren't any packets, ntop
   doesn't count things.

   ntop does not sample.  It processes every packet it sees and counts them.
   Only if there is more traffic than ntop can handle for a long period of time
   will the packet queue hit it's limit and packets be lost.  But this is still
   not sampling.

   Make sure that there's traffic on the interface(s) you are using. You can
   use tcpdump or a similar network sniffer tool to check.

   If you are on a segmented network (i.e. switched), you may not see traffic
   that isn't destined for the ntop machine unless you configure the switch to
   set the port for the ntop host into "mirror" or "management" mode (different
   vendors call it different things, but it's a mode where ALL traffic is copied
   to a specific port, regardless of which port the destination host is on).

   If there is more than one interface in the ntop host, perhaps you aren't
   listening on the one that has traffic?  Check using ifconfig:

   eth0      Link encap:Ethernet  HWaddr 00:D0:09:77:85:B9
             inet addr:192.168.0.34  Bcast:192.168.0.255  Mask:255.255.255.0
             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
             RX packets:1105906 errors:0 dropped:0 overruns:0 frame:0
             TX packets:601935 errors:0 dropped:0 overruns:0 carrier:0
             collisions:0 txqueuelen:100
             RX bytes:119869887 (114.3 Mb)  TX bytes:112203781 (107.0 Mb)
             Interrupt:11 Base address:0xc000

   If the RX and TX numbers are increasing, this shows that traffic IS
   flowing...

   If you have an unnumbered interface (listening only), remember you need to
   use -m to tell ntop what is local and what isn't:

   eth1      Link encap:Ethernet  HWaddr 00:30:F1:54:55:00
             UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
             RX packets:1596612 errors:0 dropped:0 overruns:0 frame:0
             TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
             collisions:0 txqueuelen:100
             RX bytes:566953031 (540.6 Mb)  TX bytes:0 (0.0 b)

   You can select an interface using the '-i' flag, e.g. -i eth1 or
   -i eth0,eth1.


Q. ntop doesn't understand xxxxx?
A. True. IP Packets have a source address & port and a destination address &
   port... you MUST get your head out of the application layers and revert to
   that simple concept.

   How does Apache handle virtual hosts? It analyzes the flow at the
   application level (layer 4) not the wire/packet/protocol (layers 1, 2 and
   3).  It does this by re-assembling packets into a layer 4 message (e.g. GET
   http://virtual.host.name.com/page.html)...

   Now there are some layer 4 analysis routines - virtual hosts was added in
   2.2 (and the folks who have virtual hosts have been pretty pleased), ftp, 
   http, and some others - mostly looking for traffic on non-standard
   ports, etc.

   So, since ntop works at the packet level, it doesn't understand virtual
   hosts. Unless it's SPECIFICALLY coded for.  ntop is a NETWORK analyzer, not 
   an application level one.


Q. tcpwrappers doesn't work
A. Oh yes it does... for http: connections

   1) You have to configure it this way before compiling ntop:
        ./configure --enable-tcpwrap

   2) You must have the headers and libraries installed on the build machine
      (and on the execution machine if they aren't the same).

   Remember to make the appropriate entries in hosts.allow (e.g. 
   ntop:192.168.0.) and hosts.deny (e.g. ntop:ALL)

   However, tcpwrappers and https:// is known not to work - see docs/KNOWN_BUGS


Q. My filter doesn't work!  I'm running ntop like this:
      /usr/local/bin/ntop -u nobody -L -d -E -w 3000 \
                      -m 192.168.10.0/24,xxx.xxx.xxx.xxx/32 \
                      -M -i eth0,eth1 \
                      (src net 192.168.10.0/24 or src host xxx.xxx.xxx.xxx ) \
                       and not dst net 192.168.10.0/24
A. Yup, it doesn't work.  Use the -B option and put the filter in quotes:
      -B "(src net 192.168.10.0/24 or src host xxx.xxx.xxx.xxx ) and 
           not dst net 192.168.10.0/24"

   ntop used to assume anything it didn't recognize was a filter.  But not since 
   2.1.3.  If you try this now, you should see a log warning that says maybe you
   forgot the quotes


Q. I have experienced problems defining multiple filters: ntop reports 'syntax
   error'
A. If you believe the filter is syntactically correct then it's likely that the
   Libpcap you have used has been compiled using an old non-reentrant version of
   flex.  Please make sure you're using version 2.5.4 or above.

Q. Can you give some additional examples of filters?
A. man tcpdump -- see "expression"

   A couple of simple examples, courtesy of B. Loic:

       -B "host not 192.168.1.100 and not 192.168.1.101"

   to exclude hosts 192.168.1.100 and 192.168.1.101 from tracking (FQDN 
   such as www.yahoo.com will work too). 

   If you need to exclude a full IP range, you will want to use something like

        -B "net not 192.168.1.0/24" 


Q. What about the backtrace?
A. Sadly, probably useless.  Why?  Too much information isn't available:

  Here's one (and a pretty good, obvious one at that):

      ntop[23720]:   **FATAL_ERROR** RRD: caught signal 11 SIGSEGV
      ntop[23720]:   **FATAL_ERROR** RRD: BACKTRACE: backtrace is:
      ntop[23720]:   **FATAL_ERROR** RRD: BACKTRACE:  1. /usr/bin/ntop [0x42028c48]
      ntop[23720]:   **FATAL_ERROR** RRD: BACKTRACE:  2. /usr/bin/ntop(getopt_long+0x43) [0x420c4e83]
      ntop[23720]:   **FATAL_ERROR** RRD: BACKTRACE:  3. /usr/lib/ntop/plugins/rrdPlugin.so(rrd_update+0x61) [0x44157f7d]
      ntop[23720]:   **FATAL_ERROR** RRD: BACKTRACE:  4. /usr/lib/ntop/plugins/rrdPlugin.so [0x441481a2]
      ntop[23720]:   **FATAL_ERROR** RRD: BACKTRACE:  5. /usr/lib/ntop/plugins/rrdPlugin.so(updateCounter+0x19) [0x44148a15]
      ntop[23720]:   **FATAL_ERROR** RRD: BACKTRACE:  6. /usr/lib/ntop/plugins/rrdPlugin.so [0x4414ba49]
      ntop[23720]:   **FATAL_ERROR** RRD: BACKTRACE:  7. /lib/i686/libpthread.so.0 [0x41138941]
      ntop[23720]:   **FATAL_ERROR** RRD: BACKTRACE:  8. /usr/bin/ntop(__clone+0x3a) [0x420da1ca]

    See: It doesn't give the routines, just the hex offsets from some points in the executable.
    It seems to say that rrdPlugin's updateCounter called rrd_update which called getopt_long. But
    then getopt_long called something else and that's what died...  plus we don't see the parameters.

    Sometimes it works great for me on my box, because I can use the tools to figure out precisely
    where it means.  But often I use gdb too.  Move off my box and the odds of your compile and my 
    compile (and our ./configure and all the tool versions) matching so that the offsets mean something
    are pretty small.


Q. What do these log messages mean?

     **ERROR** mVLAN: Host (identical IP/MAC) found on multiple VLANs
     mVLAN: ntop continues but will consolidate and thus probably overcount this traffic
     mVLAN: Up to 10 examples will be printed
     mVLAN: Host 192.168.aaa.bbb (00:12:17:xx:yy:zz) VLANs 3 and 4

A. Once again, they mean what they say.  ntop found something pretty weird.  It found packets
   for the same 'host' (that is the same IP and MAC address) on two different VLANs.  In this
   case, IP 192.168.aaa.bbb, MAC 00:12:17:xx:yy:zz on VLAN 3 and VLAN 4.

   It's not WRONG, but it's definitely odd.  If we see a packet twice, we count it twice, so -
   via this message - we're warning you that we might be double counting some traffic.  We warn
   you ONCE per host (per ntop run) and for only up to 10 hosts, so as not to innundate you with
   meaningless warnings.

   On the about page is a count (if you've even seen this junk), in the "Host/Session counts - global"
   section.  If it's just a couple, ignore it.
 
   If you see a lot of this stuff, you may want to rethink your ntop sensor placement or at least
   understand what you are doing.  On my home LAN, I use a 802.1q VLAN trunk to move traffic between
   two switches and have a passive tap on the trunk. Traffic to/from the firewall moves over this trunk,
   so I see the same packet on the YELLOW(DMZ) VLAN and the GREEN(TRUSTED) VLAN all the time.
 
   Why even bother trapping and warning?  Well, there is one bad thing from ntop's perspective with 
   duplicated hosts, which would occur if we didn't trap this, and that is that you would also
   see a lot of RRD errors when ntop tries to update the same host twice:
 
   **WARNING** RRD: rrd_update(/usr/.../ipBytesSent.rrd) error: illegal attempt to update using time 
                    1109523297 when last update time is 1109523297 (minimum one second step)


--------------------------------------------------------------------------------
-----Running - Web Server-------------------------------------------------------
--------------------------------------------------------------------------------

Q. Why does ntop display bits of my web site, instead of its own pages?
A. ntop is designed to search the current working directory for data files,
   such as the html subdirectory, before it searches the default directories.

   This is a feature.

   You are one of those rare souls who happen to have had an unrelated
   subdirectory 'html' with a file named 'index.html' as a subdirectory 
   of the current working directory at the time that you launched ntop. 
   cd to an acceptable directory, such as /usr/share/ntop, before 
   launching ntop.  


Q. What are High/Medium/Low risk flags
A. They are set in reportUtils.c based on fairly self-obvious functions
   Well documented in the help.html page, reachable by clicking on
   the problem descriptions or via http://127.0.0.1:3000/help.html

   Often seen if you are monitoring a backbone or common network (high)
   or if you have cloned MAC addresses for, say, a home Firewall box.


Q. What does the "Users" flag mean on a host?
A. If you go to the "Info about host xxxx" page, there will be data
   in the "Known Users" section, if it's acting as a server for certain
   protocols.

   In sessions.c, the function updateHostUsers() is used to maintain the list
   of "users" of a host.  In handleSession(), as part of the protocol level
   analysis, the "user" information for various protocols is pulled out of the
   packets.  Stuff like the "X-Kazaa-Username" header, the "MAIL FROM:" header,
   etc.

   We tag users as one or more of the following types:
        P2P_USER, SMTP_USER, FTP_USER, POP_USER, IMAP_USER

   Note that for P2P, we also record - where possible - whether this user is
   in P2P_UPLOAD_MODE and/or P2P_DOWNLOAD_MODE.


Q. Why are some of the host names in different colors?
A. Colors are used on several of the ntop pages to convey extra
   information to the user.  (in particular the ACTIVE TCP SESSIONS
   and the LOCAL HOST STATS pages). There are five colors used to
   depict how long ago the host was first seen by ntop.

   The pages which display these colors use a html style sheet called
   style.css located in the normal html subdirectory (where ntop is
   installed). This happens by setting the 'class=' parameter of
   the html 'A' (Anchor or hyper-link) tag.  The style sheet defines
   the following:

      Age of host  'class' name  Color code         Color description
      (minutes)
      -------------------------------------------------------------------
      0-5          A.age0min    { color:#FF0000 }   Red
      5-15         A.age5min    { color:#FF00FF }   Fuchsia/Magenta
      15-30        A.age15min   { color:#FF7F00 }   Coral (lt orange)
      30-60        A.age30min   { color:#007FFF }   Slate blue
      60+          A.age60min   { color:#0000FF }   Blue

   The color legend is displayed on the About | Configuration page
   (info.html).


Q. What does the P2P flag mean on a host?
A. If ntop knows enough to tag you as a P2P user, it's also looking at the
   other headers to see if it can track what files you're exchanging.  If a host
   (i.e. a workstation) downloads a file from another host ("server"), the file
   name is recorded in the list ntop maintains for both of them.

   If a host has at least one file name recorded, it's tagged with the "P2P"
   flag.


Q. What does a "Virtual Host" mean.
A. If a single instance of a web server handles many web sites, all of the
   references resolve to the same name.  The web server uses the "Host:"
   header to determine which "index.html" page to serve up.

   ntop monitors port 80 (http:) exchanges and looks for the Host: which allows
   it to build a list of virtual hosts being handled by the web server.


--------------------------------------------------------------------------------
-----Running - Web Server (https:)----------------------------------------------
--------------------------------------------------------------------------------

Q. SSL is not working! I have the following error in the log/terminal:
     10/Jun/2002 22:58:17 Started thread (6151) for network packet sniffing on
          eth0.1700:error:140EC0AF:SSL routines:SSL2_READ_INTERNAL:non sslv2
          initial packet:s2_pkt.c:187:
A. You forgot to put https:// instead of http:// in the URL you put in your
   browser!


Q. Unable to find SSL certificate 'ntop-cert.pem'
A. ntop looks such file under the current working directory, then /etc or in
   Whatever directory you configured with ./configure.

   If you want a personal certificate, you need to create it by:

      >make ntop-cert.pem

   It should be installed as part of "make install".  If you have a special
   Certificate or it's not present, do it (one-time) manually:

   For example to install it under /usr/local/etc, do:

   mkdir /usr/local/etc
   cp /usr/local/bin/ntop-cert.pem /usr/local/etc/ntop

   See docs/README.SSL


Q: What is the ssl watchdog?
A: Short answer: There are reported problems w/ the ntop web server hanging when
                 accessed via ssl (https://) from Netscape 6.2.2 (Win2K) (and
                 others).

                 The ssl "watchdog" keeps an eye on the web server - it waits
                 for 3 seconds and then if the SSL_accept call (openSSL) hasn't
                 finished, it aborts it. This leaves the user with nothing on
                 their web browser, but at least ntop's web server continues on.

                 There is no known way to send something back to the user. DON'T
                 EVEN ASK. It's not in ntop, it's the browser-server handshake
                 that's hung.  So, it looks - to the user - like a failed 
                 connection.  S'be'it...

   If you are using https:// and seem to have the problem, run ntop with the 
   --ssl-watchdog command line parameter... The item to look for on the
   configuration page (info.html) is:

       # HTTPS Request Timeouts

   Or messages in the log:

   ...: SSLWDERROR: Watchdog timer has expired. Aborting request, but ntop
        processing continues!

   You can also enable it via a ./configure parameter (./configure --help |
   less) if it's something you're going to always require.


Q. Tell me more
A. The problem is that ntop's web server is single threaded until we determine
   that the request is simply one that will be reading data.  At that point we
   fork to generate the page.  But the basic "accept a request" code is single
   threaded.  This happens all but instantaneously and hasn't been a problem
   previously.

   The code is pretty basic and pretty common:

       select() to wait for a connection, then
       ssl_accept() to fire up a "server", meaning the ssl handshake.

       Then process the http request (i.e. the GET and associated headers).

   With Netscape 6.2.2 (and others), there seems to be a bug in the Netscape
   code (ntop's is identical to other projects like sshd).

   According to something I read - but now can't find again - Netscape doesn't
   accept a legal combination of options on the handshake back from openSSL and
   hangs in a deadly embrace.  Supposedly openSSL 0.9.6c (or was it d - it's not
   in the changelog) built in a patch.  However, I didn't find the new version
   changed the behavior.

   There is stuff about a bug w/ Netscape 4.x on the openSSL website, but I'm
   not having trouble with Netscape 4.x.

   I don't understand the details and really don't care to find out.  It boils
   down to a hang in a call, SSL_accept() that doesn't have a timeout parameter.
   Argh

   Because the code is invasive, I built it (like the SIGPIPE stuff) so you can
   turn it on at ./configure time:

      --enable-sslwatchdog    Watchdog for ssl hangups (Netscape 6.2.2)
                              [default=disabled]

   or via a command line option:

      --ssl-watchdog          Use ssl watchdog (NS6 problem)

   With the "fix", ntop's web server hangs for at most 3 seconds, then continues
   on.  The user gets nada - and I don't know a way to send them anything,
   because we haven't retrieved the request yet nor done the handshake (so there
   isn't a TCP connection!)

   It only affects https:// requests and I've coded the watchdog so it doesn't
   activate unless we have openSSL and either the compile or runtime parameter
   set.  If you don't get https:// requests, it's just another idle thread.

   The fix is working for me... What I've tested (and the results with and
   without the watchdog):

    Win2k
      MS Internet Explorer 5.5 - ok
      Netscape 4.61 - ok
      Netscape 4.79 - ok
      Netscape 6.2.2 - user gets no response
                     - old: ntop webserver hung and must restart ntop!!
      Opera 6.03 - user gets a partial response
                 - old: browser says "setting up secure connection" and
                        never continues, but ntop's webserver is ok
                        (SOMETIMES you get SSL errors in log, esp.
                        if you cancel the browser)

    Linux
      Konqueror 2.2.2 - ok
      Mozilla - 1.0 - ok
      Netscape 4.78 - ok
      Galeon 1.2.5 - almost complete response, browser session is toast
                     (must restart)
                   - old: user gets nothing, but the ntop webserver is ok
      Opera 6.0B1 - user gets a partial response, but browser session is ok
                  - old: browser says "setting up secure connection" and never
                         continues, but ntop's webserver is ok.


--------------------------------------------------------------------------------
-----Running - Web Server - Security--------------------------------------------
--------------------------------------------------------------------------------

Q. I'm being prompted for a userid/password, what do I enter.
A. The default admin userid is 'Admin' (without the quotes) and the password
   is whatever you set on the 1st run of ntop (look in this FAQ for
   --set-admin-password).


Q. Why create Userids (beyond the Admin id created by --set-admin-password)
A. Multiple users allow you to control who can alter ntop's performance and/or
   view specific information. If you look on the "Admin" tab, you will see that
   you can create additional users and also control which URLs can be executed
   by whom.

   Userids could allow, for example, an ISP to allow users to access SOME
   network performance statistics, but not the proprietary stuff...

   Suppose you want to restrict who accesses the Multicast statistics page,
   multicastStats.html.

   ntop uses terminal wildcards matching the names, so multicast is treated as
   multicast* and matches multicastStats.html plus any other name beginning
   multicast...

   howto:

   1st add a new user
   2nd add "multicast" to the list of controlled screens and allow admin
       and the new user to access it (note the * wildcard is automatically
       added)

   Try and access the screen and you are prompted for a userid/password...

   Look in http.c for all the names and #defines used...

Q. So, How do I restrict access to the main http or https ntop web page?
A. To stop everyone from logging into ntop, do the following:
   Select ADMIN tab
   Select URL's then "Add Url"
   Don't fill in anything (the wildcard * is implied)
   Select only the users who you want to authorize (hold control key and
   click on user to add more users if you added users)

   and click on "Add Url"

   You will see URL '*' is added, e.g.

     'showU*'
     '*'
     'shutdown*'
     'deleteU*'
     'modifyU*'

   Then only users who know the user id and password (remember to keep the .db
   file secure!) will have access.

Q. How good is the default security ntop provides through the web server.
A. Good question...

   The default ntop configuration is not appropriate if you put ntop in a
   publicly reachable location.  We assume that ntop is either running in a
   small, trusted, LAN or that you've used other tools such as firewalls to
   protect the ntop web server.

   The userid/password scheme is good enough to prevent you from accidentally
   shutting down ntop or getting into 'dangerous' places.  But that's really
   all it does.

   Also, the security of the ntop web server is only as good as the security of
   the passwords file, ntop_pw.db - only the ntop -u userid should be able to
   read from or write to it.

   Read man crypt for information on the security of the encrypted password.


Q. The plugins aren't very secure.
A. True.

Q. How do I prevent users from turning plugins on/off?
A. The default configuration of ntop does not protect the plugin pages - no
   password is required to access showPlugins.html.

   This allows any user who can connect to the ntop web server to view reports
   FROM the plugins, but also allows them to make plugin configuration changes.

   This may not be desirable.  You may wish to add additional URLs to the
   Default list of those which require entry of a userid/password.

   You can prevent unauthorized individuals from turning plugins on/off by
   adding this URL: "showPlugins.html?" to the list via Add URL.

Q. Ok, but they can still get into the configuration pages and change things.
A. Yes.  Add the following URLs to the controlled list:

        plugins/sFlow*
        plugins/netFlow*
        plugins/rrdPlugin*

    This will keep everybody who doesn't know the userid/password out of the
    configurable plugins.  Unfortunately, it will also prevent them from seeing
    the rrd graphs, because those are created out of the rrd plugin.

A. Instead of plugins/rrdPlugin*, create these:

        plugins/rrdPlugin?d*
        plugins/rrdPlugin?h*
        plugins/rrdPlugin?i*
        plugins/rrdPlugin?r*

   It's still not perfect, the reasons why are left as an exercise for the
   user.


Q. I created a new user, adminp for administering the plugins and ntop is
   also accepting the admin userid/password.
A. Yes, the matching of userid's isn't too swift.  It's better to make sure
   that there are NO common initial substrings among ANY of the userids.


Q: How can I use Apache (with it's security, etc.) to serve up ntop pages?
A: (Toby Johnson [public@tobiasly.com], Sun 10Nov2002)

   A while back, I had written about the possibility of configuring ntop to use
   only relative URL's, in order to facilitate proxying ntop's web interface
   through Apache. I have decided it's easier to simply use Apache's ability to
   rewrite ntop's URL's when necessary. So, based on my experience, here is a
   mini-HOWTO on how to proxy ntop through Apache.

   ------

   Proxying ntop's web interface through a secure Apache virtual host is a
   convenient way to make use of any existing security measures you may already
   have. In my case, I wanted to be able to access ntop from anywhere outside
   my LAN, but opening another port on my server for ntop's dedicated web
   server wasn't an option.

   I already had a password-protected, secure web server that I use for admin
   purposes -- I'll call it https://admin.tobiasly.com. I wanted ntop's web
   interface to appear as a subdirectory under this host:
   https://admin.tobiasly.com/ntop/ .

   Here's how to configure such a setup. Change the server names and ports to
   match your own. I'm assuming that you already have a working, secure Apache
   virtual host (using HTTPS).

   First, pick a port for ntop's HTTP server. I'll use 15123. You won't need
   ntop's built-in HTTPS server, since you're proxying its content through a
   pre-existing Apache HTTPS server. Configure ntop to start with the correct
   HTTP port, and with HTTPS disabled. Something like "ntop -d -w 15123 -W 0".
   (See the ntop man page for more startup options.)

   Now, you need to tell Apache that anything under the /ntop/ URL should be
   proxied to the ntop web server. In my case, the Apache server is running on
   the same machine as ntop, so it's just a proxy to a different port on
   localhost. In your Apache secure host configuration, add a line like this:

      ProxyPass /ntop/ http://localhost:15123/

   Now, whenever Apache receives a request for something like
   "https://secure.tobiasly.com/ntop/home.html", it will proxy this request to
   the location "http://localhost:15123/home.html". Ntop will take it from
   there, generate the web content, and pass the result back to Apache. Then
   Apache passes that result back to the original client.

   It's important to note that you don't need to open port 15123 to the
   outside, since the connection actually goes through your existing Apache
   port, and then is transparently proxied by Apache on the server itself. Of
   course, you don't even have to run ntop on the same machine; as long as the
   Apache server can connect to ntop's port, it'll work.

   This is not the same as URL redirection. As far as your web browser knows,
   everything is going through https://secure.tobiasly.com/ntop/. The Apache
   server does all the proxy work behind the scenes, and simply serves up the
   results to the requesting client. And since the "outward-facing" server is
   Apache instead of ntop, you'll be using your existing Apache secure server
   certificate, instead of ntop's ntop-cert.pem.

   Everything appears to work OK at first, but we quickly run into a problem:
   some of the URL's that ntop generates are absolute. For example, to draw bar
   graphs, ntop's web pages will request the image "/gauge.jpg". This would
   translate into "https://secure.tobiasly.com/gauge.jpg". Also, host info
   pages are absolute. If I click on the host "10.1.2.3", it tries to take me
   to the page "https://secure.tobiasly.com/10.1.2.3.html".

   This is a big problem, because unless the URL is underneath the /ntop/
   directory, Apache doesn't know that it needs to proxy the request to ntop,
   and you get broken links. Luckily, Apache has the Rewrite module that lets
   us fool with requested URL's. In order to get the required URL's rewritten,
   add the following to your Apache secure virtual host configuration:

      RewriteEngine On
      RewriteCond %{HTTP_REFERER} tobiasly.com/ntop
      RewriteCond %{REQUEST_URI} !^/ntop
      RewriteCond %{REQUEST_URI} !^/error
      RewriteRule ^/(.*)$ http://secure.tobiasly.com/ntop/$1 [L,P]

   In English, this basically says "If I get a URL request that comes from a
   page that has tobiasly.com/ntop in it, and that request doesn't begin with
   /ntop, rewrute the URL to begin with http://secure.tobiasly.com/ntop/, and
   pass this rewritten URL to the Proxy engine." At this point, the Proxy
   engine will see that it is getting a URL that begins with /ntop/, and
   correctly pass it to the ntop web server. Rewriting the request to begin
   with HTTP instead of HTTPS may seem incorrect, but since that URL will be
   handed directly to the Proxy engine, it can't be HTTPS or ntop's web server
   will not recognize it.

   Now, you should be able to simply connect to
   https://secure.tobiasly.com/ntop/ , and you're ready to go!

   NOTE courtesy of Bruno Lebayle <lebayle@esrf.fr>

   I had many problems with the "Rewrite" stuff described in the FAQ. After
   some Google search, it appears that HTTP_REFERER is sometimes not
   reliable, and the browser I am using (Firefox/Solaris) does not seem to
   present this header properly.
   So I've found an other way to do it, and this proves to work with many
   different browsers:

   within mod_proxy.c:
    ProxyPass /ntop/ http://localhost:3000/
    ProxyPassReverse /ntop/ http://localhost:3000/

   <IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteLogLevel 0
    RewriteLog logs/rewrite_log
    RewriteCond %{REQUEST_URI} !^/$
    RewriteCond %{REQUEST_URI} !^/home.html$
    RewriteCond %{REQUEST_URI} !^/ntop/
    RewriteRule ^/([^/]+\.[a-z]+)$ http://nms.my.domain:my_port/ntop/$1[L,P]
   </IfModule>

   Where http://nms.my.domain:my_port is:
   - the host where the network management tools are located with the
   proper authentication
   - the host where the above Apache config has to be set
   - the host where ntop is installed on port 3000 in this example
   Where the Web site on this host has no URL in the form of /xxx.xxx aprt
   from / and /home.html (additions can be made in the RewriteCond)

   Especially, the trailing "/" after /ntop in the RewriteCond is
   mandatory, since e.g. the Ntop logo is /ntop_logo.gif and would match
   /ntop

   # Rewrite help summary:
   # HTTP_REFERER = pattern in the name of the page from which the request comes
   #    THIS IS NOT RELIABLE AND DIFFERS AMONG BROWSERS !!!
   # REQUEST_URI = contents of the request (^ = start of line, $ = end of line,
   #    ! = not the text which follows
   # All conditions preceding a rule are evaluated
   # All rules are processed in sequence
   # Rewrite: () = group of text used for the substitution,
   #    . = any char, * = repeated 0-N times,
   #    + = repeated 1-N times, ^ = not this char, [chars] = one of chars
   #    special characters escaped by \ e.g. the dot as \.
   #    $1 means the 1st group of text between ()
   #    Flags L = last (i.e. exit from the rewriting process)
   #          P = proxy (i.e. use the proxy module for this URL)
   #
   # Full doc in http://httpd.apache.org/docs/1.3/mod/mod_rewrite.html
   # Useful help in http://rewrite.drbacchus.com/rewritewiki/

--------------------------------------------------------------------------------
-----Running - Web Server - i18n------------------------------------------------
--------------------------------------------------------------------------------

Q. Is ntop localized for language x? (i18n)
A. No.  ntop wasn't really written with i18n in mind.

   Most of the text is generated in-line, on the fly.  Plus ntop must
   dynamically support multiple locales simultaneously.

   However, beginning with v2.1.56 (2.2 development release), there is limited,
   optional, i18n support in ntop.

Q. So, what internationalization (i18n) support does ntop provide.
A. The key word is LIMITED

   This only applies to the pages that are pulled from .html files, NOT those
   created internally.  This includes the menus and the few static text pages,
   but none of the pages with interesting data on them.

   The localized pages must be placed in parallel directories to the existing
   html ones.

   For example, if ntop is installed in /usr/share/ntop, the html files are in
   /usr/share/ntop/html.

   To support them Canadians, then, you would need to create a
   /usr/share/ntop/html_en_CA AND that locale would need to be installed on the
   ntop host system.


   Note that there are NO i18n files distributed with ntop (yet!)


   At ./configure time, you enable support via --enable-i18n.  ntop MUST be told
   how to find the locale files.  In ./configure, a "standard" location is
   defined per OS.

   (Initially only the value for FreeBSD is populated).  All others assume the
   "default", /usr/lib/locale.  If that isn't right for your OS, then you MUST
   use the optional parameter --with-localedir= to tell ntop where to find the
   files.


   At run time, ntop scans the host for the installed locales (locale -a should
   - on most systems give you a list) and checks if a comparable html_cc_XX
   directory exists.

   This builds a list of supported languages, which (along with i18n status) is
   shown on the configuration pages, info.html and textinfo.html.


   When an http request is made, your browser sends a list of languages it is
   willing to accept in the http Accept-Language: header.

   (check View | Internet Options | Languages in IE to see what you're sending)

   For example,

       Accept-Languages: en_US, en

   Means that you prefer US English, but will accept any English dialect if US
   English isn't available.


   Be aware that the locale settings and Accept-Language settings are not well
   standardized, nor common and may not necessarily map very cleanly.  You
   should see what's defined (perhaps it's locale 'german' instead of 'gr') and
   make or link directories as necessary.  You can always create the directory
   you tell ntop to use via --with-localedir= in the /usr/share/ntop structure
   and create links from there to the real locale directories!


   Limits in the per-request and total # of languages to support are in
   globals-defines.h


   Because of directory structure limits, a lack of interest in multiple
   character sets, etc. the locale and accept-language headers are coerced into
   a common format:

   locales are                  ll[_XX][.char][@modifier]

       ll - language, usually the 2 character ISO abrev., such as us, it.
       XX - dialect (often a country), such as CA or US (en_US != en_CA)
       char - character set (we sort of assume UTF-8)
       modifier - euro

   Accept-Language: values are  ll-XX or ll or ll-*


   Once the user makes a request, each page pulled is checked:

       1. For each of the Accept-Language values.
       2. For the ntop host locale value.
       3. In the ntop default (English) set.

   These checks are performed for each of the libraries specified in the config
   value (CFG_DATAFILE_DIR).

Q. What about the country flags.
A. There are other sets available on the web, of different quality and size.
   Rather than chase down permissions and rights, we'll stick with what we have
   but let you know here of other options.

   If you find a set you like, just download them and replace the xx.gif files 
   in .../html/statsicons/flags

   Much better, but about 4x larger:
      http://users.skynet.be/hermandw/fl/smalgifs.html

   Same height, but wider (so they look better):
      http://www.kidlink.org/www/miniflags.html


Q. What pages can be customized?
A. ls /usr/share/ntop/html/*.html (or wherever the ntop pages are installed):

   Also, remember that a file overrides ntop's internal page generation, so you
   can also use this facility to override ANY of ntop's pages and return a
   customized page (perhaps you don't want users seeing them?).


--------------------------------------------------------------------------------
-----Running - Web Server - p3p-------------------------------------------------
--------------------------------------------------------------------------------

Q. What's up with P3P?
A. P3P is a W3C recommendation - http://www.w3.org/TR/P3P/ - for specifying how
   an application(typically a web site) handles personally identifiably
   information.  What information the site collects and what it does with the
   information.

   p3p is pretty complex!  There are basically two ways to enable an application
   for p3p. One is to add another HTTP header, P3P:.  The second is to support a
   well-known file location, /w3c/p3p.xml (like robots.txt).

   Browser support is pretty spotty, as is web site adoption.

   Some 3rd party browsers have some support...  up to CrazyBrowser which claims
   "full support", whatever that means...


Q. So why put P3P into ntop?
A. It's coming.  P3P is gradually making it's way into the top web sites -
   right now (Dec2002), for example dell.com supports it and yahoo.com doesn't.


Q. Ok, but what's that got to do with ntop?
A. Since ntop collects personally identifiable data in it's access log (-a
   option) and it's various reports and makes those available to pretty much
   anyone in the default configuration, it's probably not a bad idea to OFFER
   some support.  Especially if you're running ntop at a site that has started
   to support P3P, if you don't have a mechanism for your own policies
   you'll have to adhere to corporate ones.  And that could require massive
   changes to ntop.


Q. IE6?
A. Since ntop doesn't send the P3P: header, IE6 ignores ntop wrt p3p.  Besides,
   IE6 uses p3p to block 3rd party cookies.  If you want to see the p3p stuff,
   it's view | privacy report in the menus.  If the site's policies don't match
   your settings, there will be a red "do not enter" icon in the third box on
   the bottom right of the IE6 window - double click on it to see the report.
   See http://support.microsoft.com/default.aspx?scid=KB;en-us;q293513


Q. Mozilla
A. Unknown if it's enabled by default.  Mozilla had support, ripped it out in
   Feb 2002 and put a new version back in.

Q. Other browsers
A. See their home pages or search the web.  One that I know that claims "full
   support" (whatever that means) is at http://www.crazybrowser.com/


Q. Privacy Bird?
A. A browser-add-on, AT&T's privacy bird (http://www.privacybird.com/), that I'm
   playing with is a lot more aggressive in supporting p3p.  If Privacy bird
   doesn't see the P3P: header, it then requests the "well known" file,
   /w3c/p3p.xml file and gets nailed by ntop as a hostile application, since we
   don't have support for returning .xml files (yet).


Q. So when & how does ntop support p3p?
A. A patch in the cvs on 4Dec2002 adds minimal support for p3p -- specifically:

      1) ntop will respond to queries for /w3c/p3p.xml and ntop.p3p -- returning
         the ntop.p3p file, IF ONE EXISTS.

         If the file does not exist, a 404 error is generated (vs. pre 4Dec2002
         behavior of adding the address to the myGlobals.weDontWantToTalkWithYou
         list).

      2) New parameters, --p3p-cp and --p3p-uri allow you to return the P3P:
         header with either or both of the parameters (cp="" or policyref="")
         set.

         ntop doesn't validate the text in any way other than the usual
         stringSanityCheck().

   This allows me to run the Privacy Bird and still talk to ntop.  I'll admit
   that option #2 is speculative, since I really don't have much of a way to
   test it.


Q. But there isn't a sample .p3p file provided.
A. Right.

   Please note that there is no sample file provided.  This is not an oversight.

   After careful consideration, I am not providing one.  The reason is that a
   .p3p file is intended to be a legal contract between your site and your users.
   While I could provide a default file that has the right tags - as I
   understand p3p - for the data ntop collects and stores, I don't want the
   responsibility and/or liability.

   If anyone wants this "sample p3p file", I will make it available for a fee,
   Provided your organization - through an appropriate officer, in writing:

      1) Acknowledges that Luca Deri, Burton Strauss and other developers of
         ntop have no liability for any use(s) you make of the sample p3p file
         or anything you derive from it.

      2) You will defend us - at your expense - from any lawsuit, arbitration
         proceeding, etc. filed in conjunction with your use of the sample p3p
         file.

      3) You will pay any judgments, legal expenses, etc. related to any
         lawsuit, arbitration proceeding, etc. in conjunction with your use of
         the sample p3p file.

   Since your legal department would be nuts to agree to that I doubt it will
   come up.


Q. So How do I create a .p3p file?
A. There are tools available to create p3p policy files - search the web for
   'p3p editor'. One that I've used is a zero cost albeit beta tool, p3peditor
   from IBM (http://www.alphaworks.ibm.com/tech/p3peditor).


--------------------------------------------------------------------------------
-----Networks, Network cards and Networking-------------------------------------
--------------------------------------------------------------------------------

Q. My security people won't let me run in promiscuous mode.
A. Tough...

   Or, use the -s option and accept the limitations...

   Ask them "honestly, what is the problem" - other than having an interface
   in promiscuous mode is a signature of a sniffer and security folks look for
   unauthorized sniffers?

   ntop needs promiscuous mode so that it sees the full range of traffic.  Any
   similar product will do the same thing.

   If the security people think traffic on the wire is secure, they're wrong!
   Face facts - just about every Windows user, except for 2K/XP Pro (and then
   only if TBTP have especially locked them down) can install the windows
   version of tcpdump...

   If it's a checklist item, just gen up a form to "authorize" it, have the
   boss and VP/CIO sign it and give it to them.


Q. What is Ethernet and TCP/IP and how do they differ?
A. Both are protocols - that is the definition of how
   to interpret bits on wires (or in packets) into
   meaningful conversations.

   Ethernet is the lower level, wire (or wireless) protocol,
   concerned with moving the physical bits of data.

   TCP/IP is the higher-level protocol, which explains
   how to interpret the block of bits (frame).

   TCP/IP uses a familiar 32-bit "IP" address, e.g.
   192.168.0.1.

   Ethernet uses a less familiar, 48 bit unique to the NIC
   (some times called "burned in") address, e.g.
   00:40:05:DE:AD:00.  This is called the MAC (Media
   Access Control) address.

   FYI: The official IEEE MAC address lookup is at
       http://standards.ieee.org/regauth/oui/index.shtml
   (Look up the first six digits, separated by -s, e.g. 00-40-05)


Q. OK, but how is stuff sent from my computer to, say, Yahoo!?
A. First off, your computer does a lookup - using a service
   called DNS (Domain Name Service) to convert www.yahoo.com
   to a numeric value, such as 66.218.71.80.

   Then it builds a collection of characters that says send
   this data from me, 192.168.0.1 to Yahoo at 66.218.71.80.
   This is called a packet.  That gets wrapped in an Ethernet
   frame (addressed from 00:40:05:DE:AD:00 to the MAC address
   of the local gateway router, 0:d0:9e:6:38:00 and squirts it
   out the router.

   Packets are forwarded step by step along a path from you
   to Yahoo by computers called routers.  This is done based
   on the 32 bit IP address and the router's knowledge of the
   network.

   Each router sees a Ethernet frame addressed to it (by
   MAC address), checks the TCP/IP address to figure out
   where to send it next, re-wraps the TCP/IP packet in a new
   Ethernet frame (with the from MAC as it's own and the to
   MAC as the next hop).

   This happens until the TCP/IP packet reaches the final
   segment (the last router).  Once it reaches a router that
   knows it has addresses 66.218.71.0-66.218.71.255 on one
   of it's interface, the routing stops using the TCP/IP
   address.

   The last hop is done (like each intermediate hop - at the
   lowest level) based on the MAC address!  Specifically, the
   last router does an "ARP" (Address Resolution Protocol") query,
   to find out "Who Has" address 66.218.71.80.  The NIC responds
   with it's MAC address:

      arp who-has www.yahoo.com tell router
      arp reply www.yahoo.com is-at 0:d0:9e:6:38:00

   And the packet is routed to that address.

   Alright, that's a bit simplified, but see Douglas Comer,
   "Internetworking with TCP/IP, volume I", page 25 and 73ff.


Q. Tell me more.
A. OK, gang time to teach Ethernet & TCP/IP basics one more time.
   With pictures...

   Suppose you have a network that looks like this (we'll use 
   impossible addresses 288 - just pretend it's ok):

              (ext) 288.1.1.1     (int) 288.2.2.1
                            +-----+
     World+Dog ------------ + ISA + ------- LAN ----- WS 288.2.2.2
         |                  +-----+
         |                     |(dmz) 277.1.1.1
        me 299.0.0.1           \----------- DMZ ----- MAIL 277.1.1.2
                                                  \-- WEB  277.1.1.3

   ISA can be acting solely as a router or it can be acting as a NAT device.
   That's irrelevant, so we assume it's not.

   I send you a packet.  It travels the Internet and arrives at your 288.1.1.1
   the ISA(router) with src=299.0.0.1, dst=288.2.2.2.

   Like every router along the way, ISA(router) looks at the destination
   address and realizes it has to route the packet on to 288.2.2.2.  So the
   ISA(router) sends the packet on, out the best interface to reach 288.2.2.2.

   Remember, however, the TCP/IP packet is wrapped in a lower level (Ethernet)
   packet at the wire level.  Read your TCP/IP and Ethernet standards - the
   actual delivery of packets over links is this Ethernet level and it uses the
   48 bit MAC address.

   This "Ethernet" packet is actually what travels hop to hop to hop (you can
   even see these headers if you have visibility to the traffic - it's called
   the link level header by tcpdump and you'll see the 48 bit MAC addresses if
   you use the -e parameter).

   In order to be able to handle the Ethernet level signaling, each router
   rewrites the packet so that the 48 bit source MAC address it it's own
   (from-router that is) and the destination MAC address is the one that
   from-router has in it's tables for to-router (the next hop).

   So the packet looks like this, where the srcMAC and dstMAC get rewritten
   each hop, so that the routers on both ends of the link know whom it's
   addressed to:

   Hop1 (srcMAC=00:00:00:aa:aa:aa dstMAC=00:00:00:bbbbb:bb frame=IP
   data=(src=299.1.1.1 dst=288.2.2.2 data="Hi!")
   Hop2 (srcMAC=00:00:00:bb:bb:bb dstMAC=00:00:00:cc:cc:cc frame=IP
   data=(src=299.1.1.1 dst=288.2.2.2 data="Hi!")
   Hop3 (srcMAC=00:00:00:cc:cc:cc dstMAC=00:00:00:dd:dd:dd frame=IP
   data=(src=299.1.1.1 dst=288.2.2.2 data="Hi!")

   Notice how the TCP/IP stuff isn't changed.  But the MAC address is.

   At each hop, the NIC card, operating at the Ethernet level, sees its own MAC
   address and knows to accept the packet. It passes it up the protocol stack,
   where the next layer (TCP/IP) realizes it needs to be routed further on...

   Ultimately, the packet gets delivered to some service listening on your WS.

   Here's a packet capture to show you:

   # tcpdump -Xx -c1 -i eth0 -e
   tcpdump: listening on eth0
   11:49:10.809890 0:3:47:b1:xx:xx 0:e0:18:b4:yy:yy ip 118:
   tigger.ssh > zebra.2714:
   P 1824243567:1824243631(64) ack 2328789523 win 11792 (DF) [tos 0x10]
   0x0000   4510 0068 b305 4000 4006 b1e4 c0a8 2a24        E..h..@.@.....*$
   0x0010   c0a8 2a21 0016 0a9a 6cbb bf6f 8ace 8213        ..*!....l..o....
   0x0020   5018 2e10 ee1a 0000 469a 3e34 eda7 549e        P.......F.>4..T.
   0x0030   0ec4 4847 8983 fb4f 65ea 5c3e 0bbe c325        ..HG...Oe.\>...%
   0x0040   7db8 9954 dae1 55b6 54f9 cdfd ac07 a2b5        }..T..U.T.......
   0x0050   ce4f

   So this says, the packet came from tigger (MAC address 0:3:47:b1:xx:xx) ->
   zebra (MAC address 0:e0:18:b4:yy:yy)
     Within that is the tcp/ip packet, from c0a82a24 -> c0ae2a21
   (192.168.42.36 -> 192.168.42.33)

   Here's another one, from tigger -> router.

   # tcpdump -Xx -c1 -i eth0 -e host 192.168.42.1
   tcpdump: listening on eth0
   11:52:48.712750 0:3:47:b1:aa:aa 0:d0:9e:6:bb:bb ip 72:
   tigger.32782 > homeportal.gateway.2wire.net.domain:
   41356+ A? cvs.ntop.org. (30) (DF)
   0x0000   4500 003a bf7e 4000 4011 a5be c0a8 2a24        E..:.~@.@.....*$
   0x0010   c0a8 2a01 800e 0035 0026 ce2e a18c 0100        ..*....5.&......
   0x0020   0001 0000 0000 0000 0363 7673 046e 746f        .........cvs.nto
   0x0030   7003 6f72 6700 0001 0001                       p.org.....

   (0035 = port 53, so it's a dns query)

   And one more, from cvs.ntop.org -> tigger (which has to have passed through
   the router)

   # tcpdump -Xx -c1 -i eth0 -e "not src net 192.168.42.0/24"
   tcpdump: listening on eth0
   11:53:39.688806 0:d0:9e:6:bb:bb 0:3:47:b1:aa:aa ip 69:
   195.31.151.66.cvspserver > tigger.42964:
   P 2566885448:2566885451(3) ack 2903504154 win 24616 <nop,nop,timestamp
   389837493 85533859> (DF)
   0x0000   4500 0037 5f3c 4000 2906 ad56 c31f 9742        E..7_<@.)..V...B
   0x0010   c0a8 2a24 0961 a7d4 98ff 9048 ad0f f51a        ..*$.a.....H....
   0x0020   8018 6028 279a 0000 0101 080a 173c 72b5        ..`('........<r.
   0x0030   0519 24a3 6f6b 0a                              ..$.ok.

   See the MAC address?  It's the routers, not cvs.ntop.org's

Q. Why do I care?
A. Because that's sometimes a problem you'll have with ntop. See ntop is 
   nosy, so it puts the interface into promiscuous mode, where every 
   packet - addressed to the ntop host or not - is processed.  Now the 
   next layer up, the tcp/ip layer will throw away any 'junk' (You can 
   just hear it going tisk, tisk, tisk).  But libpcap can intercept them 
   at the Ethernet level and feed them to ntop...

   For REMOTE hosts, ntop uses the IP address as it's the only valid data.
   But for LOCAL hosts, ntop prefers to use the MAC address as a way of
   resolving multi-homed or multiply addressed hosts.

   See if you have two IP addresses assigned to the same host on your local
   network, say 192.168.42.42 and 192.168.42.43, how is ntop to tell they're
   the same host?  Well, the MAC addresses are the same...  (for some
   manufacturers, e.g. Sun, ALL of the interfaces on a host use the same MAC
   address).

   read getHostInfo() in hash.c.

      /* This is a local address hence this is a potential
         multihomed host. */

      if((el->hostIpAddress.s_addr != 0x0)
         && (el->hostIpAddress.s_addr != hostIpAddress->s_addr)) {
         isMultihomed = 1;
         FD_SET(FLAG_HOST_TYPE_MULTIHOMED, &el->flags);
      }

   i.e. if the address we've stored for this host doesn't match this one, 
   it's multihomed.

      } else if((hostIpAddress != NULL)
                && (el->hostIpAddress.s_addr == hostIpAddress->s_addr)) {
         /* Spoofing or duplicated MAC address:
            two hosts with the same IP address and different MAC
            addresses
          */

         if(!hasDuplicatedMac(el)) {
            FD_SET(FLAG_HOST_DUPLICATED_MAC, &el->flags);
            ...
         }

         setSpoofingFlag = 1;
         hostFound = 1;
         break;
      }

   If the addresses DO match, we've had two MAC addresses, so this is being
   spoofed.

   etc.


Q. So why do I get bad output?
A. If, somehow, you've confused ntop - for example telling it that 
   277.1.1.0/24 in the ascii art example (above) is local, then ntop 
   is going to believe you.  And it will see a packet with the 
   277.1.1.1 IP and a MAC address. And use that.  Only it's not the MAC 
   address of the MAIL host, it's really the MAC address of ISA.

   No matter, ntop doesn't know this -- all it sees is the packets and 
   the data you gave it.  So later on, when it sees a packet with the 
   same MAC address, but a different IP, well, it will assume that it's 
   the same host... and the data will all be lumped together.


Q. Does that explain why I'm seeing xxxx as multihomed?
A. Maybe.  Remember - it only takes one packet, not even an ack, for
   ntop to create a host record.  If that's wrong, it will carry 
   forward - and you'll probably see the host tagged as 'Multihomed'
   when correct packets show up.  Say:

      Host 1: IP 192.168.1.1 MAC 00:00:00:aa:aa:aa
      Host 3: IP 192.168.1.3 MAC 00:00:00:cc:cc:cc

   If somebody has the incorrect hosts table, dns, cached, whatever 
   but has the info that Host 1 is 192.168.1.3 and is on the same 
   subnet, then it will send a packet where the Ethernet layer and
   the ip are nonsense.  But because it's on the same wire, the ip 
   is ignored:

   (Ethernet from:00:00:00:dd:dd:dd to:00:00:00:aa:aa:aa)
       (tcp s=192.168.1.4 d=192.168.1.3)

   ntop will read both out of the packet and create the association

      192.168.1.3=00:00:00:aa:aa:aa

   Since it doesn't know better.

   Then when it sees

   (Ethernet from:00:00:00:ee:ee:ee to:00:00:00:cc:cc:cc)
      (tcp s=192.168.1.5 d=192.168.1.3)

   It will create the multihomed association...


Q. So what's a hub vs. a Switch
A. A hub is a device that links a bunch of computers together
   at the wire (Ethernet) level.  Logically, Ethernet is a bus,
   that is everybody sees all the traffic, just like cars crossing
   under a highway bridge.   Physically, Ethernet is wired like
   a star - with all the wires coming back to a central "hub".
   The hub is just the device that makes the electric star look
   like a shared bus.

   Switches and Hubs operate at the Ethernet level, not TCP/IP.

A. Watch out for 'Switched hubs', which are hubs that include an
   internal switch between 2 or more segments (for example, BUT
   NOT LIMITED TO a 10BaseT and 100BaseT) segment.  These are hubs
   within a segment, but switches across segments.  ntop may not
   see the traffic you expect if you have a 'switched hubs' and
   manufacturers are pretty bad about marking them. See
   http://article.gmane.org/gmane.linux.ntop.general/5081


A. A switch is a smart hub.

   Switches improve performance by creating a virtual Ethernet
   bus for the duration of the packet that joins JUST the source
   and destination ports.

   A switch operates via an internal table of MAC addresses.
   It learns (or is programmed) that 0:d0:9e:6:38:00 is on
   port 1, while 00:40:05:DE:AD:00 is on port 3.

   A packet coming in port 1, destined for 00:40:05:DE:AD:00
   is sent out ONLY port 3.

   If the switch doesn't know (or the packet is a broadcast),
   it gets sent out all ports.

   This doesn't make for MORE bandwidth, but it does use it
   more efficiently.  That is in addition to the session between
   ports 1 and 3 at 100Mbps, a second, simultaneous 100Mbps
   session can occur between ports 2 and 4.


Q. How do I use ntop in a switched network?
A. First off, you need to be or have the support of
   your network administrator.  (Yes, you can do something
   called "ARP poisoning" to - maybe - get the switch to send
   you all the traffic, but that's beyond this FAQ... STFW)

   Many switches (although not the USD$50 cheap "workgroup" units)
   have a special port or mode, where by all the traffic for the
   entire network gets copied out that port, in addition to the
   normal switch action.

   When you invoke the monitoring mode (called span, mirror, monitor,
   analysis, etc.), you are forcing the entire switch bandwidth out one
   port.  This may exceed the bandwidth of the port.  100Mbps+100Mbps
   >> 100Mbps!

   Traffic that is being sent to the monitoring port in excess of the
   capacity of that port is usually dropped.  It should NOT slow down
   the switch on other ports.

   Some switches have some buffering capability and it *may* be able to
   keep up with an occasional burst of traffic, as long as the average
   is below the port capacity and the buffer isn't exceeded.

   See, for example, http://www.cisco.com/warp/public/473/41.html#archXL.

   One list of switch manufacturers is the document is titled "REFERENCE:
   Configuring a Switch to Monitor All Traffic" from Elron Software. (The
   URL is long, do a Google search for "site:elronsoftware.com wi6038").


--------------------------------------------------------------------------------
-----Dumping data---------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Can I use ntop from php/perl?
A. Yes you can. Please see the www directory under the ntop source tree.
A. Look at the Admin | Dump Data page.


--------------------------------------------------------------------------------
-----rrd (myrrd)----------------------------------------------------------------
--------------------------------------------------------------------------------

Q. How do I save data between runs?
A. Use rrd.


Q. What's rrd?
A. There's a 12 page writeup on what rrd is and what ntop does with it.
   A pdf version is posted at SourceForge in the "Documentation" release.
   Download it and read it.


Q. Do I gotta?
A. Yup - the pretty pictures won't work in this FAQ.


Q. I'm lazy - What is rrd?
A. RRD stands for "round-robin database". It is a special type of database designed for
   holding sequences of information over periods of time, without growing in size. 

   Rrdtool is a tool for manipulating RRDs. The home page for rrdtool is at 
   http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/ 

   What do they do? Well, suppose you want to compute the average of the traffic to your
   web site for the last fifteen minutes. If you record the data each minute and save it
   in a traditional database it looks like this: 


                10:45   1.00 MB
                10:46   1.02 MB
                10:47   0.27 MB
                ...
                11:00   0.54 MB
                ...

   While you have the data to compute the total, the database grows in size forever. And
   all you really need are the last 15 values. Certainly you can create a purge routine
   and periodically remove the old data. But this type of constantly growing SQL database
   - even with a prune process - will require reorganization and rebuilds over time. 

   Suppose you had some kind of data structure where the last value was thrown away each
   time you added a new one. When it comes time to store the 11:01 value, you overlay the
   10:46 value. At any time, you still have the last 15 values. That - slightly simplified - 
   is an RRD. The benefit is that your database never grows in size. The down side is that 
   everything else in your history is gone - if your needs ever change, tough. 

   The ring buffers are called round-robin archives or RRAs. The RRA actually stores the 
   RATE (bits per second), so the 10:47 value of 0.27 Megabytes is 0.27 * 1024*1024 * 8 / 60
   or 37748.736 (bits per second). But it functions just like the rings described above. 

   ntop uses RRDs to store data over long periods of time. Separate files are created for 
   each counter, in a structure that reflects the interfaces and hosts ntop sees. The specifics 
   of what's recorded - interfaces or not, hosts or not, etc. is controlled by switches on the
   ntop rrd plugin. 

   (Read the paper - it goes on from here, with specific details on how ntop uses RRDs). 

Q. What's myrrd?
A. ntop includes a frozen and slightly patched version of rrd 1.0.49 in the
   ntop source tree.  This is called myrrd.

   rrdtool 1.0.49 was released 08-Aug-2004.
           1.0.50 was released 25-Apr-2005.

   rrdtool 1.2.x began to be released in Apr 2005.


Q. What's rrdtool? 
A. rrdtool is the packaged program to access the various rrd routines (export, dump, graph)
   from the command line.

Q. How do I get it?
A. The myrrd version included with ntop doesn't have rrdtool. Assuming you haven't installed 
   an rrdtools package (which ntop ignores), here is how to get rrdtool: 

   The myrrd version is a frozen copy of rrdtool 1.0.49, so your best bet is to go to the 
   rrdtool home page, http://www.rrdtool.com/ or the download page, 
   http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/pub/ and download rrdtool-1.0.49.tar.gz. 

   Once you have it, it's pretty standard. 

   $ tar xfvz rrdtool-1.0.49.tar.gz $ cd rrdtool-1.0.49 $ ./configure 

   (You can add --prefix=/usr if you want) 

   $ make 

   This runs a few minutes. 

   Now you can do 

   $ make install 

   but remember, ntop just ignores this, so why bother? All you really need are the rrdtool
   executable program (src/rrdtool) and maybe a few of the files in the doc/ directory. Just
   copy them where you want them and then delete the whold build directory. 


Q. Where do I get rrd?
A. Unless you need rrdtool, you should not need to do anything to get rrd (see myrrd, above).

   The home page for rrd is
   http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/

   rpm's are available at
   http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/pub


Q. What about the multi-threaded development version?
A. Stay away.

   (UPDATED) I was able to patch ntop to work with 1.2.7+ and experimented with it a little bit.
   
   The binary .rrd file formats are different, so if you try 1.2.x, any new .rrd files which are 
   created are incompatible.

   The new version doesn't use freetype, it uses libart, which introduces a different (not better,
   nor worse) chain of dependencies.


Q: I've enabled the rrd plugin and there's no data ... there are messages in the log:

        RRD call stack:
        argv[0]: rrd_update
        argv[1]: ...rrd/matrix/12.239.98.199/12.239.181.175/pkts.rrd
        argv[2]: 1037289548:1
        rrd_create(...) error: creating '...': No such file or directory
        rrd_update(...) error: opening '...': No such file or directory

A: Create the rrd directory and make sure that the -u userid has read/write
   access to it (typically /usr/share/ntop/rrd).


Q: Still nothing...
A: Remember to activate the plugin.  You will need to configure it, and remember that
   the default configuration does not include per-host data.


Q. What's the difference in the Host Detail Level for RRDs?
A. Low is just the bare counts, pktSent/pktRcvd and bytesSent/bytesRcvd.

   Medium adds:

     pktDuplicatedAckSent/pktDuplicatedAckRcvd,
     pktBroadcastSent, bytesBroadcastSent,
     pktMulticastSent, bytesMulticastSent,
     pktMulticastRcvd, bytesMulticastRcvd, 
     bytesSentLoc, bytesSentRem, bytesRcvdLoc, bytesRcvdFromRem, 
     ipBytesSent, ipBytesRcvd, 
     tcpSentLoc, tcpSentRem, tcpRcvdLoc, tcpRcvdFromRem, tcpFragmentsSent, tcpFragmentsRcvd, 
     udpSentLoc, udpSentRem, udpRcvdLoc, udpRcvdFromRem, udpFragmentsSent, udpFragmentsRcvd, 
     icmpSent, icmpRcvd, icmpFragmentsSent, icmpFragmentsRcvd, 
     ipv6Sent, ipv6Rcvd

    NonIP:

     stpSent, stpRcvd,
     ipxSent, ipxRcvd,
     osiSent, osiRcvd,
     dlcSent, dlcRcvd,
     arp_rarpSent, arp_rarpRcvd, arpReqPktsSent, arpReplyPktsSent, arpReplyPktsRcvd,
     decnetSent, decnetRcvd,
     appletalkSent, appletalkRcvd,
     netbiosSent, netbiosRcvd,
     otherSent, otherRcvd

    And the per-protocol Sent/Rcvd

   High adds:

     totContactedSentPeers, totContactedRcvdPeers

    And the per-IP-protocol Sent/Rcvd, e.g. IP_HTTP...


Q. rrdPlugin - problem with rrd/myrrd?
A. By default, ntop's Makefile binds the static libmyrrd library to create ntop's 
   rrdPlugin shared library.  THIS IS DELIBERATE so that you use the myrrd library
   and not some other version of rrd that's installed somewhere else on your system.

   Sometimes this causes problems, where there are special tricks required to tell
   (a non gnu ld) loader about static (.a) libraries, which ntop doesn't have in
   the Makefile nor the configureextra files.

   As a SHORT TERM WORK-AROUND, you can TRY this:

      $ cd myrrd

   Edit Makefile --

       all: $(LIBRRD) libmyrrd.so
                      ^^^^^^^^^^^ add this 

   Add these lines:

       libmyrrd.so: $(OBJECTS)
       <tab>ld -shared -o libmyrrd.so $(OBJECTS)

   Now do make.  You should see a libmyrrd.so file.

   The main 'make' should now complete.  Copy that libmyrrd.so file where the other
   ntop library files are, and it MIGHT work.

   However, the whole idea behind having a static libmyrrd.a is to prevent version
   conflicts and use a stable version of rrd.

   The right fix is to get the configureextra/<whatever> file changed.


--------------------------------------------------------------------------------
-----netFlow and sFlow----------------------------------------------------------
--------------------------------------------------------------------------------

Q. How do I access netFlow or sFlow data from ntop?
A. You need to configure ntop as a listener.

   First, use the appropriate plugin to set the parameters - basically the port
   you want ntop to listen on.  Then, using the Admin | Set Interface menu item,
   switch ntop to report on the sFlow/netFlow pseudo-device (NetFlow-device or
   sFlow-device).


Q. Can I use ntop as a netflow collector.
A. Not in the current versions - you used to be able, but that code was removed.


--------------------------------------------------------------------------------
-----netFlow--------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Which versions of netFlow.
A. v5
   And v1/v7/v9 - in that internally a v1/v7/v9 flow is copied to a v5 buffer and then
   processed.  We default/ignore fields that are different.
   And nFlow - similar conversion.


Q. netFlow doesn't work.
A. You MUST make sure the ntop plugin is ACTIVE.  With the change to allow
   setting parameters while inactive, it's easy to miss that last step.  
   If you don't activate the plugin, you'll still have the netflow-device, but
   no data on it...


Q. What's Virtual NetFlow Interface?
A. Be sure and set it.  It's important for pseudo-local classification, which
   affects L R reporting.  You need to set it to the (network) and mask for 
   the netFlow collector.  So ntop knows 'where' the data is coming from.

Q. 'splain some more, Lucy...
A. OK.

   It's best to think of netFlow like this:

   The physical interface which is monitoring the packets is like a
   temperature probe you stick into a roast.

   Even though the display of the data can be right there at the probe, or
   the other end of a (long) wire, or somewhere entirely elsewhere via a 
   wireless connection, the probe is monitoring at the tip.  If it says 145F,
   that's the temperature of the meat - not the oven and not the kitchen.

   Similarly, the netFlow data ntop is receiving is based on the probe
   location.

   So, if you have a router and are monitoring a single interface to collect
   netFlow data, then the ip address you want to give to ntop is that of
   the router interface.

   If you are monitoring a router with more than one interface, you will 
   need to give ntop ONE of those addresses and use the -m | --local-subnets
   option to tell it that the other addresses are also local.


Q. Where is info about netflow?
A. Dale Reed pointed out a good tech doc (no flak, just the formats) for netflow
    V1/5/7:

http://www.cisco.com/univercd/cc/td/doc/product/rtrmgmt/nfc/nfc_2_0/nfc_ug/nfcform.htm

   (As of Oct2003, it includes v8 and is here:)
http://www.cisco.com/univercd/cc/td/doc/product/rtrmgmt/nfc/nfc_3_0/nfc_ug/nfcform.htm


Q. How Do I Enable NetFlow Data Export on a Cisco Device?
A. To enable netFlow Data Export (NDE) from a Cisco device to an ntop netFlow
   receiver on port 2055 (default) at address 10.1.1.1:

     ip flow-export destination 10.1.1.1 2055
     ip flow-export version 5

   You may want to designate the source interface, e.g.:

     ip flow-export source Ethernet0

   Enable netFlow on each interface to be monitored. netFlow normally only
   captures data from each incoming packet, so to see traffic in both directions
   netFlow must be enabled on both the incoming and outgoing interfaces. As an
   example, for an Internet access router this would mean enabling netFlow on
   both the internal (e.g. Ethernet) and the external (e.g. ISDN / Frame Relay
   etc) interfaces:

     interface Ethernet0
     ip route-cache flow

     interface Dialer1
     ip route-cache flow


   By default netFlow will only export flow statistics shortly after the flow
   Terminates or when 30 minutes have elapsed. In many environments, you want
   ntop to be a bit more up to date. To change the timeout to five minutes:

     ip flow-cache timeout active 5

   The following 'show' commands are useful for examining netFlow statistics
   directly on the Cisco box and may assist when setting up ntop:

     show ip flow export
     show ip cache flow
     show ip cache verbose flow

   Obviously, there is a lot more to it than this, for more information, see the
   Cisco web site: http://www.cisco.com/go/netflow
                                     (Created by sholmes at snapshot, 02Feb2003)

Q. Can I run ONLY w/ netFlow (Running ntop as a Collector for Net Flow only)
A. Sure.

   ntop is usually configured to capture all traffic from local interfaces. If no 
   interface is given (e.g. option -i|--interface is missing), the "first" interface 
   is taken. This value, typically eth0 on linux boxes, may not be what you wanted. 

   If you want to use ntop as a collector for Net Flow traffic only, you may want to
   supress all local traffic. 

   In this case use -i none or --interface none. 

   Remember to activate Net Flow plugin from Menu Admin->Plugin and to configure the
   plugin by setting "Local Collector UDP Port". 

--------------------------------------------------------------------------------
-----sFlow----------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. What is sFlow
A. The core component of the sFlow toolkit is the sflowtool command line
   utility. sflowtool interfaces to utilities such as tcpdump, ntop and Snort
   for detailed packet tracing and analysis, NetFlow compatible collectors for
   IP flow accounting, and provides text based output that can be used in
   scripts to provide customized analysis and reporting and for integrating with
   other tools such as MRTG or rrdtool.

   Some info:

   http://www.inmon.com/sflowTools.htm
   http://www.faqs.org/rfcs/rfc3176.html

Q. I have activated the sFlow plugin in ntop. But it doesn't seem to
   generate any output based on the collected sflow datagrams.
A. sFlow can be a collector or a receiver or both, depending on the
   settings configured via the plugin.

   If you configure ntop as an sFlow collector, it will use sFlow data
   for generating reports, treating the remote collector(s) as another
   network interface - see Admin | Switch NIC.

Q. sFlow doesn't work.
A. Check this out:

     This talks about a bad experience I had setting up sFlow reception. For the longest
     time, I could see that ntop was getting sflow packets, but no data would show up. 
     It turns out the switch I was exporting from didn't see any real traffic, and it was 
     just sending COUNTERSAMPLE packets..... 
 
     - - - - - - - - I figured out that it was indeed "invalid" sflow packets. 

     Apparently, sflow sends COUNTERSAMPLE and FLOWSAMPLE packets. COUNTERSAMPLE packets 
     give a quick look at interface counters on the machine, whereas FLOWSAMPLE packets 
     are actual packet fragments from IP connections. Ntop seems to simply parse, 
     debug_print, and discard COUNTERSAMPLE packets...which made it confusing to look at 
     the debug output and say "wow, lots of sflow coming in!" when in fact it was just for
     show, as Burton suggested. I added more switches (with active connections) to the
     switches sending sflow packets and I now have hosts with pretty graphs. 




--------------------------------------------------------------------------------
-----Solaris--------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. How do I install the ntop package on Solaris?
A. For instance do 'pkgadd -d ntop-2.2-solaris.i386'


Q. What c compiler do I need?
A. Sun's cc or gnu's gcc.

Q. What about /usr/ucb/cc is that the one?
A. No.  cc is software you pay for.  /usr/ucb/cc is a stub compiler and good for,
   recompiling the kernel and absolutely nothing else.

   The cc I mean is Sun's Commercial Compiler.


--------------------------------------------------------------------------------
-----BSD - FreeBSD--------------------------------------------------------------
                                        During the ntop 3.2 development cycle,
                                        we did development/testing under:
                                           4.10 and 4.11
                                           5.3 and 5.4
--------------------------------------------------------------------------------

Q. When I type 'make' it complains about a makefile error.
A. Always remember to use gnu make.

   On many *BSD systems which have other 'make's, gnu make is called gmake.
   Try make --version -- if it shows a Gnu version stamp you're ok, otherwise
   try gmake.


Q. I get "ntop: /dev/bpf0: Device not configured", what's wrong?
A. This is because bfpX has not been configured inside the generic bsd-kernel
   config file.

   If you use generic kernel config file put "pseudo-device bpfilter 16" in
   kernel config file and rebuild the kernel.


Q. I remember rumors about something not being right under FreeBSD with
   threads?
A. Yes.  See FreeBSD bug bin/17437 at 
   http://www.FreeBSD.org/cgi/query-pr.cgi?pr=17437

   Basically, due to limits in FreeBSD, there is no pthread_atfork() function.
   So, when ntop does it's fork() call to create http pages, it can't fixup
   the Mutexes.  It wrong and could conceivably cause problems.

   However - ntop ran for years without the pthread_atfork() code, so we're
   no worse off in 3.0 under FreeBSD than in 2.2 or 2.1...

Q. The web server problem?
A. There was a flurry of problems late in the 3.0 development cycle having to
   do with a seeming deadlock of the ntop web server (it's actually not dead,
   just walking at about 0.001KPH).

   Thanks to Yeoman efforts by Stanley Hopcroft, Michal Meloun and, well, me,
   we have a work-around.  

   With 3.1 we tried to automate this workaround, but fell a foul of FreeBSD's
   fixes.  So in 3.2 we've reverted to requiring the command line flag.

   If you're running under FreeBSD and have problems, use the flag, --set-pcap-nonblocking.

   For more on this, read the threads at gmane - look for "FreeBSD and pthreads" - 
   that's probably the best summary.  But there's stuff on this back at least to 
   October 2003 - look for Stanley's problems with CPU usage.

A. Also, understand that --set-pcap-nonblocking is going to increase ntop's
   cpu usage.  It will probably come close to pegging the CPU at 100%.  Yet
   strangely other processes won't seem to be impacted. (Of course, you really
   should be running ntop on it's own host, anyway, right?).

   This is because of how the work-around is coded. ntop should step aside
   briefly and let them run.

   Just be aware of it and don't ask on the mailing lists.

   It's not giant lock vs. fine grained, etc., it's really the handling of 
   signals (user interrupts) that is different. 

   Say you have a process, A running.

   In linux, POSIX thread x is kernel process x' and POSIX thread y is kernel 
   process y'.  Now the POSIX standard doesn't prevent thread y from receiving 
   a signal 'intended' for x, so you have to code for that.  But when a thread 
   'sleeps', the process sleeps, which is something the kernel understands and 
   so the kernel can do other work.

   In a userland threads model (i.e. xBSD), POSIX thread x and POSIX thread y 
   are both parts of kernel process A.  To implement the POSIX threading, the
   thread library code creates a thread manager, thread z'. That thread receives
   all signals, interupts etc. and parcels the work back out.  

   This too is perfectly within the POSIX model.

   BUT: To implement the thread manager implies that the user task (A+x+y+z) is 
   active whenever it's waiting for ANYTHING.  How well this works depends upon 
   the sophistication of kernel processes that recognize when the user task is 
   'idle', meaning it is spinning waiting for somethings, vs. doing useful work.

   In FreeBSD 4.x, we see that this recognition is not very sophisticated when 
   it comes to the bpf pseudo driver and so ntop will use 100% of the available CPU.
   Even so, the scheduler bounces between tasks of equal priority, so productive
   work does get done, but you see the high CPU usage charged to your task.


--------------------------------------------------------------------------------
-----Linux - all----------------------------------------------------------------
                                        During the ntop 3.2 development cycle,
                                        we did development and/or testing under:
                                           Gentoo 2005.0
                                           Fedora Core 2 and 3

                                        The cvs version was run by users on many
                                        other versions.

                                        Others should certainly work and there
                                        are many user reports of success.
--------------------------------------------------------------------------------



--------------------------------------------------------------------------------
-----Linux - RedHat-------------------------------------------------------------
--------------------------------------------------------------------------------

Q. "application bug: ntop(...) has SIGCHLD set to SIG_IGN but calls wait().
   (See the NOTES section of 'man 2 wait'). Workaround activated."

   This message and the NOTES section of the man page lead me to believe
   that the problem is handled but the kernel feels the need to report it
   from time to time.
A. Read the NOTES section.  The problem has been handled, but the kernel
   feels the need to report it from time to time.  See the article at
   http://article.gmane.org/gmane.linux.ntop.general/5304


Q. libpng version conflicts?
A. See libpng, below.

--------------------------------------------------------------------------------
-----Win32 - Common-------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Where can I find GDBM for Windows?
A. GDBM for windows can be found at http://www.roth.net/libs/gdbm/


Q. ntop -i1 ... doesn't work
A. ntop has special parameters under Win32

         Under win32 there are TWO COMPLETELY SEPARATE TYPES OF PARAMETERS.

     There are the parameters to the win32 stub AND there are parameters to ntop
     itself.

     AFTER THE win32 parameters are the ntop parameters in the standard (Unix)
     -xxx format.

         ntop /c <normal parms>  runs ntop INTERACTIVELY with the specified ntop
         parameters

         ntop /i <parameters> installs ntop as a service to run with the specified parameters

         ntop /r removes the ntop service

   Remember, ntop /i and ntop /d don't actually run the service - you need to
   start it.

   REMEMBER: there are TWO ways to run ntop, one as a service, one 'interactively' (/c).

   They are totally separate.  Just because you ran ntop w/ interactively with some parameter 
   set, does not affect the stored parameter set for the service.

   To change the stored parameters, reinstall the service vi /i.


Q. How do I figure out what my network interface numbers are for the -I
   parameter?
A. (Thanks to jac engel [jacengel@home.nl] for the example)

   If you only have ONE network interface, it doesn't matter as the default is
   fine.  However, that's the RARE case.  Most people have multiple network
   interfaces (NICs), with virtual ones for VPNs, Dialup Networking, etc.

   The Windows tools ipconfig, winipcfg and the Device Manager (depending on
   which version of Windows you have) will probably show you them.  However,
   it's easier and better to use ntop to show you how ntop sees the network
   interfaces.

   If you start  ntop /c (interactive mode, with only the default parameters) it
   Will display all your network interfaces (NICs), like this:

    Running ntop for Win32.
    Wait please: ntop is coming up...
    23/Aug/2002 20:43:55 Initializing IP services...
    23/Aug/2002 20:43:55 Initializing GDBM...
    23/Aug/2002 20:43:55 Initializing network devices...
    23/Aug/2002 20:43:55 Found interface [index=0] '\Device\Packet_{14...1C}'
    23/Aug/2002 20:43:55 Found interface [index=1] '\Device\Packet_{86...B4}'
    23/Aug/2002 20:43:55 Found interface [index=2] '\Device\Packet_NdisWanIp'
    23/Aug/2002 20:43:57 ntop v.2.1 MT [WinNT/2K/XP] (11/07/2002 build)
    23/Aug/2002 20:43:57 Listening on [3F1C}\Device\Packet_{14...1C}

  By default, ntop will use the lowest numbered interface.  Because #s are
  assigned based on the sequence cards are discovered, and this is altered if
  cards are removed and added, this is often not what you want.

  After you figure out which NIC you want, start ntop /c -i1 or -i2 or
  whatever...


Q. OK, but how to I translate \Device\Packet_xxxxx to my Froboz ModelT network
   card and not the Fubar27 that's on the motherboard.
A. ntop should report both the index and the human readable information.
A. A Google search on script "CurrentVersion\NetworkCards" finds a couple of
   scripts/utilities that might work in various environments.

A. Otherwise...

   You're going to need to view the registry.  All the usual warnings - back up
   your pc, etc.

   If you damage the registry, you may not be able to reboot the computer.

   You're not going to CHANGE anything, but an inadvertent keystroke could be
   disaster ... BE CAREFUL!

   Under WinNT/2K, to find the interface name of your NIC look in the registry
   at the keys in 
     HKLM\Software\Microsoft\Windows NT\Currentversion\NetworkCards\
   The two subkeys, Servicename and the Description tells you which id maps to
   which NIC.


Q. Where does ntop look for html (and gif) files under Win32?
A. ntop looks in two places. The first is the current directory and the second
   is configurable through a constant in ntop_win32.h, #define DATAFILE_DIR "."

   Note that the current directory, or ".", may not be what you expect.

   When running ntop as a Win32 service, "." is %SystemRoot%\system32, meaning
   that ntop looks in %SystemRoot%\system32\html for the .html and .gif files.

   When running ntop from the command line,

       ntop /c parameters...

   "." is whatever directory is current.  This means that if you run ntop with a
   full, explicit path (c:\ntopnew\ntop /c ...) there may be an unexpected
   difference between what ntop finds for "." and what you THINK "." is!  This
   will lead to missing .html and .gif files.

   If you wish to have ntop look in a specific place for the files, the best
   choices are:

     1) Create a .bat file to run ntop which does a cd to the expected directory
        first.
     2) Edit ntop_win32.c and then recompile.

   Note that the settings for DATAFILE_DIR (and other constants) are reported on
   the text version of the configuration page, textinfo.html.



--------------------------------------------------------------------------------
-----Win32 (MinGW) (Windows)----------------------------------------------------
--------------------------------------------------------------------------------

Q. What's the scoop with ntop on Windows?
A. Semi-officially,

   ntop for Windows 95/98/ME/NT/2K is also provided as a binary application with
   limited capture capability (1000 packets). This is intended to allow 
   demonstration of ntop for people without access to a Unix system.

   We call this version Win32 after the old official name of the Windows library.

   If you want to use the full version with unlimited packet capture you can
   either:

       * Recompile ntop from the source by yourself (Luca says just open the
         files in MS Visual C++ 6.0 and press compile)
       * Register your ntop for Windows 95/98/NT/2K copy by paying a 
         convenience fee to receive the prebuilt executable.

   If you decide to register your copy, Luca will send you an URL from which
   you  can download the full version periodically.


Q. So where did MinGW come in?
A. All of the necessary open software tools have been ported to run under
   windows (sort of), so it is theoretically possible to build and run ntop
   under Windows.  

   However, Windows and *nix are very, very different internally.  So we have
   to use special versions of the packet capture library (winpcap instead of
   libpcap).  And we needed our standard tools (gcc et al), plus some 'glue'
   so we could make *nix calls and have Windows things happen.

   For the tools and glue, there are three choices: native, Cygwin and MinGW.

   Native means you put lots of #ifdefs in to make Windows calls where you
   had *nix calls.

   Cygwin is a shim - it's a Windows dll (Dynamic Link Library) that pretends
   to support the *nix calls and then does the right Windows things for you.

   MinGW is a project to create native Windows tools and executables from
   *nix code.

   Each of these have limits, pluses and minuses.  For example, Cygwin's dll
   has caused all sorts of problems (dlls and their *nix equivalent, shared
   libraries, usually do).  MinGW has some limits.  As MinGW grew, ntop for
   Win32(MinGW) got closer to ntop under *nix.  Native means supporting two
   'separate' code bases.

   Luca actually picked a hybrid - he uses Microsoft Visual C++ 6.0 - which
   has it's own (albeit incomplete) shim layer.  As long as you stay within
   the limits of the shim, the same code works across platforms.  In other
   places you have to make thinks like Windows wants them.  How close are
   we?  grep for WIN32.

   The advantage of the hybrid was that is was also pretty close to working
   under MinGW - creating native executables from the base code using free
   tools.  So people sent in patches and it pretty much worked.  Up until
   2.1.3, that is.  But it was never officially more than an 'it also runs'.

   After 2.1.3, Luca embarked on a process of bringing the WIN32 and *nix
   code closer together.  Surprisingly, this actually broke MinGW!


Q. Where do I get MinGW?
A. The MinGW home page is http://www.mingw.org.  Quoting:

      "MinGW is a collection of header files and import libraries that 
      allow one to use GCC and produce native Windows32 programs that do
      not rely on any 3rd-party DLLs. The current set of tools include 
      GNU Compiler Collection (GCC), GNU Binary Utilities (Binutils), 
      GNU debugger (Gdb) , GNU make, and a assorted other utilities. We
      are currently working on creating a complete set of Mingw-hosted
      GNU toolchain, and looking for volunteers."


Q. Does ntop work under Cygwin?
A. The .exe distributed through ntop.org is built with Visual C++ 6.0.
   It proved just barely possible to use the same code under MinGW.
   Forget about cygwin.


Q. Does ntop (v3.2) work under MinGW?
A. Maybe (I think) -  During the 3.1 development cycle changes were made to
   support MinGW and it should again work - see docs/BUILD-MinGW.txt


--------------------------------------------------------------------------------
-----Win32 (MS Visual C++)------------------------------------------------------
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
-----Libpng---------------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Bad things - I see the following messages:
        libpng warning: Application was compiled with png.h from libpng-1.0.x
        libpng warning: Application is running with png.c from libpng-1.2.x 
        gd-png: fatal libpng error: Incompatible libpng version in application
              and library
A. You have a version problem with libpng.

   First off, following the instructions in BUILD-NTOP.txt should work just
   fine. These problems come about when you have libpng installed (i.e. using
   shared libraries).

   1. If you are compiling from source, you may have png.h left over from the
      earlier version of libpng. Remove it.

   2. (Most common under RedHat). RedHat 7.2 installs a libgd.so.1.8.4 library,
       which was compiled against 1.0.x series of libpng (which is fine, because
       RedHat 7.2 includes libpng-1.0.12).

   Updating RedHat to newer (RawHide) packages for libpng,
   http://www.rpmfind.net//linux/RPM/rawhide/1.0/i386/RedHat/RPMS/libpng-1.2.2-5.i386.html,
   should work. However, there are reports of version conflicts and required
   updates to multiple packages. Proceed with caution (especially if you decide
   to uninstall 1.2.2-5).
   Also, do not use --nodeps or --force, as this can leave you with two
   partially installed versions (see item #1, above).

   3. (Slackware) Users have reported this error from an older header file in
      /usr/include.
   Make sure to run "make install" in the libpng directory so that the latest
   files are in the common library locations. You can do this with buildAll.sh,
   just navigate back down to the libpng-1.2.x directory first.

   4. If you are building ntop on one machine and running on another, they may
      have different libpng.so versions.  Even if you think you are using the
      static linked version (buildAll.sh), be careful - see the entry (above) on
      "make install" for libpng.

Q. When I run ./configure, it finds png.h but not libpng:
      *******************************************************************
      *
      * ERROR: libpng header or library routines are missing
      *           (yes means it was found, no means it was not found)
      *
      *              png.h...yes
      *              png_read_info() in -lpng...
      *
      *>>> No way to proceed.
      *
      *???        Install libpng (and/or libpng-devel), check www.libpng.org
      *???    and Rerun ./configure
      *
      *******************************************************************
A. You're missing libpng.so.  Look for it (locate libpng) and tell ntop
   where it is via the --with-png-lib= parameter.


Q. I seem to be missing libpng.so - when I do locate libpng, it finds:
      /usr/lib/libpng.so.3
      /usr/lib/libpng.so.3.1.2.2
      /usr/lib/libpng12.so.0
      /usr/lib/libpng12.so.0.1.2.2
      /usr/lib/libpng.so.2.1.0.13
   but no libpng.so...

A. Yeah.  Send RedHat a nasty gram.

Q. I don't understand...
A. In a normal libpng install, say from the source, you would have - in 
   addition to the .so.n.n.n.n files - a symlink named libpng.so, like this:

      lrwxrwxrwx 1 root  root   19 Apr 23 15:46 libpng.so -> libpng12.so.0.1.2.2

   But, that link seems to be missing.  Without it, -lpng doesn't properly
   resolve and you get the ./configure error.

Q. Why?
A. RedHat ships Linux with both versions of libpng, a 1.0.x and a 1.2.x version.

   Do this:

      $ rpm -qa | grep libpng

   And you'll see the libpng 1.0.x run time:

      libpng-1.2.2-8
      libpng-devel-1.2.2-8
      libpng10-1.0.13-6

   Dig into the files installed by them and you'll not find libpng.so.

   Since they're incompatible RedHat doesn't create the libpng.so link.
   Instead they patch the makefiles to point the various packages at one or the
   other .so file they install.  This allows them to ship packages that require
   one or the other.

   It works fine unless you try and install something other than via an rpm.
   Then you're missing the libpng.so file that normal packages look for...

   Best bet is to create a symbolic link from the libpng.so.xxxx installed 
   by the package which matches the -devel (because that's where png.h is 
   found), e.g.:

   ln -s /usr/lib/libpng.so.3.1.2.2 /usr/lib/libpng.so

   And remember this in case you update the libpng rpm's in the future.


--------------------------------------------------------------------------------
-----Silly Season---------------------------------------------------------------
--------------------------------------------------------------------------------

Q. Who is Pixel
A. My cat.


--------------------------------------------------------------------------------
-----HowTo Ask For Help (ntop mailing lists)------------------------------------
--------------------------------------------------------------------------------

HOWTO ask for help on the ntop or ntop-dev mailing lists:

WHERE TO POST
=============

ntop is for user questions - "How to I install", "data isn't being recorded",
etc.

ntop-dev is for code and development questions.  The ntop-dev list goes to fewer
people, those who have self-selected themselves to be interested in ntop at the
code level.

ntop-misc is for other products - nProbe, nBox, PF_RING, etc.

Discussions about the current cvs version belong in ntop-dev.

If a discussion gets too technical, you may be asked to "move it to ntop-dev".
Please honor that request (even if you have to subscribe for a while - ntop-dev
is fairly low traffic).

There used to be mailing lists and trackers at SourceForge, which were rarely
looked at and have been discontinued. Use the ntop and ntop-dev lists (go to
http://www.ntop.org to signup for them).


OFFICIAL vs UNOFFICIAL
======================

A response from Luca Deri should be considered official.  He is the author of
ntop and controls the project and it's destiny.

***Please understand that the mailing lists are a community support effort***

Besides Luca Deri, a number of people answer questions to the best of our
ability. None of the rest of the people who may respond to your question on ntop
or ntop-dev are able to respond "officially".

Likewise, this HOWTO is unofficial.

Everyone is welcome to help with the evolution of ntop - that is to find
problems, create and test patches and send them in to patches@ntop.org for
inclusion.  There are a small number of people with write access to the cvs,
but anything we commit is subject to being ripped out by Luca for any reason...
or no reason at all...


QUESTION FORMATS
================

ONE QUESTION per message, and you MUST use meaningful message subjects - one's
that would have helped YOU find the prior discussion of this or a similar
problem in the archives.  Titles such as "urgent" or "ntop problem" will often
not get a response - it may be urgent to you but it's probably not an issue
for others.

WE STRONGLY SUGGEST YOU USE THE BUILT IN AUTOMATICALLY GENERATED PROBLEM
REPORT (About | "bug icon").  This includes most of the internal configuration
data we ask for (and more) and has blank spots for you to fill it.

Generate one, cut & paste into your mail client, edit the data and sent it.

Beyond that, don't worry -- it's about information, not format.


RESPONSES
=========

Despite any individual's frequent postings, nobody is "responsible" for
answering your question. It's all on a "best efforts" basis. Our responses may 
be incomplete, inaccurate, even dead wrong. Caveat Emptor! The only "guarantee" 
is that free support will be worth what you've paid for it.  It may be worth MORE, 
it won't be worth LESS.

Just because you post a question does NOT mean that you are OWED an answer.

If nobody answers, then maybe it's because:

   * Nobody knows.
   * People are busy.
   * You've asked the same question multiple times and it's already been
     answered.
   * You have been asked for additional information and are unable/unwilling
     to supply it.

or, well, any one of a dozen other reasons.

Asking the same question multiple times - or asking it again because you don't
like the answer you received - is a slap in the face of the person who took the
time to answer you in the first place and will more than likely not get a
different response.  If you're not sure that your message posted, check the
archives to see if your message is there -- please don't just keep reposting it.

You can always use gmane (http://www.gmane.org) to see the last 600 or so
postings to the lists.

Please direct all original postings and subsequent replies to the list, not to
someone privately.  Most of us will reply solely to the mailing list, unless
you specifically request otherwise.  If you do request otherwise, the individual
you sent it to may choose not to respond.  Our posting here is NOT a public
invitation to invade our e-mail boxes for your free private support.


THE BACK AND FORTH PROCESS
==========================

"Why don't you just fix my problem instead of asking for more information?"

Understand that we can't see your machine (and wouldn't want the
responsibility of sshing into somebody else's box as root).  The only
information we have is what you post and the responses to our questions.  Few
failures in ntop are related to the core processing routines - so if you're
having a problem, it's most likely because of some combination of your network
and your ntop configuration.  It may be unique to you -- and only with YOUR help
can it be resolved.


WHAT IS SUPPORTED?
==================

Releases are hosted at SourceForge.

At the time of this writing the stable version is 3.0. What support is available
is for the development version ("the cvs").  All support is in the form of
fixing things in the cvs.

However we also attempt to support the current "stable" release (3.0).

Older versions are not supported -- especially 1.3 and the 2.0.99 series of
2.1 release candidates.  If you have a problem with them, please obtain the
current cvs version and see if it's still a problem.  Unlike certain much larger
projects, we don't fix things in older versions - there simply aren't enough
resources available.

intop is not supported.  It's gone and no longer in the cvs.  For a look at
Rocco's new, work-in-progress, download ntcsh, the ntop enabled tcsh shell.

Please understand that the only way to fix your problem may be a source patch,
which you will have to apply, compile, install and test against the cvs version
prior to it's inclusion in the cvs.

If you aren't capable of or willing to do these steps -- for whatever reason --
then you should not be compiling from the cvs.


CVS
===

The cvs is at http://cvs.ntop.org, userid is anonymous, password ntop.

The cvs is a DEVELOPMENT version.  The code in the cvs is subject to rapid
change.  At any point in time, it may not compile.  It may not compile with
certain options or on some platforms. s'be'it -- it's a DEVELOPMENT version.


3.1
===

The actual flow of ntop development was 3.0.50.. ->
                                    -> 3.0.50.. -> 3.1rc1 -> 3.1


3.2
===

The actual flow of ntop development was 3.1.1.. -> etc (I think .4 was the last Win32 version),
                                    -> 3.1.50.. -> 3.1.51 -> 3.2rc1 -> 3.2rc2 -> 3.2

FEE-BASED SUPPORT
=================

If you want better than "best-efforts" support, contact the individual you
Desire support from off-list to make financial arrangements.  Please understand
that people are doing development in areas that are of personal interest to
them, to improve ntop.
If you want to discuss payment for support or a specific change that is of
Interest to you, feel free to email the individual off-list - some of us are
Computer consultants and can be bought, with the understanding that the work
product is offered back to the community in the spirit of the open source
movement and the strictures of the GPL.


SO WHAT INFORMATION SHOULD I POST?
==================================

BEFORE POSTING:

1. Please review the output from ./configure.

    We all have the bad habit of skipping over this, but there are often
    warnings which explain why things don't work.  ntop tries to build itself by
    turning off features where the required libraries and/or headers aren't
    available.
    The minimum required set is just that - minimal.  This is often the source
    of "feature x or switch y doesn't work" reports.

2. Please review the docs/FAQ file.

3. Please review back message traffic from the mailing lists.

     Yes, we know that there isn't a search function at ntop.org.  Did you know
     that the lists are spidered every couple of months or so and can be
     searched through Google??  For example, "site:lists.ntop.org rpm" will find
     mail list messages with the word "rpm" in them.

     Do you know about gmane (http://www.gmane.org) has archives (searchable) of
     the ntop lists going back into late 2001.  The lists are called

          gmane.linux.ntop.devel
          gmane.linux.ntop.general

     You can read these online (the last 600 messages or so) or through the nntp
     server.


POSTING:

Do not worry about posting TOO much information - we're pretty good at filtering
out the noise.

WE STRONGLY SUGGEST YOU USE THE BUILT IN AUTOMATICALLY GENERATED PROBLEM
REPORT (About | "bug icon").  This includes most of the internal configuration
data we ask for (and more) and has blank spots for you to fill it.

Generate one, cut & paste into your mail client, edit the data and sent it.

If you can't or won't use the automated problem report (say, for example you
can't get ntop up to generate it) - don't worry -- it's about information, not
format.  Send us what you can, organized this way:

1. A brief summary of the problem.

2. Operations
     The EXACT command line you use to invoke ntop.
          If it's in a script, cut & paste it and
          resolve all the variables!

     Error Messages:  Cut & paste the exact text.
          If it's in the log, give us 15 or 20 lines before.

     The exact URL you used from the browser.

3.  Software
     ntop version, source and any applied patches

            If you've compiled from the source, say so!

            If you're using a package (such as an .rpm), where
                did you get it from and what is the EXACT name,
                version information and date? (for example, post
                the output from rpm -q ntop -i)

     OS vendor & version

     gcc version (e.g. gcc --version)
       (For ./configure problems, the versions for autoconf, automake and
        libtool too)

     glibc version

     Any major upgrades (kernel, networking, etc.)

     What else is running

4. Hardware
     Type & # of processors

     Amount of memory

     # network interfaces and types (vendor, bus, etc.)

5.  Network
     Roughly where are the interface(s) you're monitoring
          (Public Internet, Private LAN, what?)

     What's the bandwidth (e.g. 10 Mbps University internet,
           1.5 Mbps T1, Cable Modem capped at 1.5Mbps, 56K dialup)

     How many machines (traffic sources/destinations) and users

(If you're uncomfortable giving specifics, then leave it generic, but the
information is necessary to allow efficient use of the community's time helping
YOU with YOUR problem)


AFTER POSTING:

Please let us know if our help fixed the problem, didn't solve it or enabled you
to solve it yourself and what the result was.  The historical record of the ntop
and ntop-dev archives is the complete chain from problem to resolution.

(Originally posted on 07Jan2002 to ntop and ntop-dev, updated.  This is version 12April2003)

--------------------------------------------------------------------------------
-----GDB ultraMini-tutorial - Running ntop under gdb (debugger)-----------------
-----or, 'capturing the failure point'
--------------------------------------------------------------------------------

The very best way to debug a segmentation fault in ntop is to use gdb. The
Standard ntop compile already has the flags necessary to do this set.

(Note - if you don't have gdb, or aren't compiling yourself, this won't work)

> gdb /usr/bin/ntop (or wherever ntop is installed)
...
(gdb) set args (your usual arg string) -K

[That is, add the -K argument. While you are at it, don't give it the -d
argument and add -u root (replace any existing -u value) - yes, it's insecure
running as root, but you're not planning on doing this in production nor as a
routine situation!]

it will run... when it bombs...

"bt full" does a decent job of printing the stack and the back trace and the
local variables at each level. 

Just make sure you are in the thread you are interested in (i.e. DO THIS FIRST)

(gdb) bt full
#0 0x40592557 in __libc_pause () from /lib/i686/libc.so.6
No locals.
#1 0x4046b5a3 in pause () at wrapsyscall.c:123
result = -1073743680
oldtype = 0
#2 0x0804ac1b in main (argc=22, argv=0xbffffa44) at main.c:928
argc = -1073743680
argv = (char **) 0x0
i = 0
userSpecified = 1
ifStr = "eth0,eth1", '\000' 
lastTime = 1025633918
#3 0x404f3647 in __libc_start_main (main=0x804a74c , argc=22, ubp_av=0xbffffa44,
init=0x8049600 <_init>, fini=0x804d000 <_fini>, rtld_fini=0x4000dcd4 <_dl_fini>,
stack_end=0xbffffa3c) at ../sysdeps/generic/libc-start.c:129
ubp_av = (char **) 0xbffffa44
fini = (void (*)()) 0x40016b4c <_dl_debug_mask>
rtld_fini = (void (*)()) 0xbffff87c
ubp_ev = (char **) 0xbffffaa0
(gdb)

Now, continue:

if there are any seemingly interesting variables involved, you can print them:

(gdb) print deviceId

[gdb can handle pretty complex arguments in the print command, so you can say
"print myGlobals.device[0].hash_hostTraffic[myGlobals.broadcastEntryIdx]"
if that's what it bombed on.]

Sometimes gdb won't find a variable, that's because C used a register. S'be'it.


Other interesting things:

(gdb) list [this shows where in the code it died]

(gdb) info threads [this shows the status of the multiple threads]

(gdb) info stack [this shows the call stack]

If you want, you can check each of the threads and THEIR status via:

(gdb) thread 1 [or any valid number]
(gdb) info stack

gdb has all kinds of included help, and frankly that's all I know...
Here are some extracts and examples:

(gdb) help info thread
IDs of currently known threads.

(gdb) info thread
  13 Thread 11276 (LWP 8967)  0x405f651e in __select () from
/lib/i686/libc.so.6
  12 Thread 10251 (LWP 8966)  0x405f651e in __select () from
/lib/i686/libc.so.6
  11 Thread 9226 (LWP 8965)  0x405f651e in __select () from
/lib/i686/libc.so.6
  10 Thread 8201 (LWP 8964)  0x405f651e in __select () from
/lib/i686/libc.so.6
  9 Thread 7176 (LWP 8963)  0x405f651e in __select () from
/lib/i686/libc.so.6
  8 Thread 6151 (LWP 8962)  0x405ca071 in __libc_nanosleep () from
/lib/i686/libc.so.6
  7 Thread 5126 (LWP 8961)  0x4053db85 in __sigsuspend (set=0x4300998c)
    at ../sysdeps/unix/sysv/linux/sigsuspend.c:45
  6 Thread 4101 (LWP 8960)  0x405ca071 in __libc_nanosleep () from
/lib/i686/libc.so.6
  5 Thread 3076 (LWP 8959)  0x405ca071 in __libc_nanosleep () from
/lib/i686/libc.so.6
  4 Thread 2051 (LWP 8958)  0x405ca071 in __libc_nanosleep () from
/lib/i686/libc.so.6
  3 Thread 1026 (LWP 8957)  0x4053db85 in __sigsuspend (set=0x4100967c)
    at ../sysdeps/unix/sysv/linux/sigsuspend.c:45
  2 Thread 2049 (LWP 8956)  0x405f4e17 in __poll (fds=0x823dbec, nfds=1,
timeout=2000)
    at ../sysdeps/unix/sysv/linux/poll.c:63
* 1 Thread 1024 (LWP 8952)  0x405ca027 in __libc_pause () from
/lib/i686/libc.so.6

The * indicates the current thread.  Switch with this:

(gdb) thread 7

Once you have a thread selected, you can look at the call stack:

(gdb) help stack
Examining the stack.
The stack is made up of stack frames.  Gdb assigns numbers to stack frames
counting from zero for the innermost (currently executing) frame.

At any time gdb identifies one frame as the "selected" frame.
Variable lookups are done with respect to the selected frame.
When the program being debugged stops, gdb selects the innermost frame.
The commands below can be used to select other frames by number or address.

List of commands:

backtrace -- Print backtrace of all stack frames
bt -- Print backtrace of all stack frames
down -- Select and print stack frame called by this one
frame -- Select and print a stack frame
return -- Make selected stack frame return to its caller
select-frame -- Select a stack frame without printing anything
up -- Select and print stack frame that called this one

Type "help" followed by command name for full documentation.
Command name abbreviations are allowed if unambiguous.
(gdb) info stack
#0  0x4053db85 in __sigsuspend (set=0x4300998c) at
../sysdeps/unix/sysv/linux/sigsuspend.c:45
#1  0x404db1c9 in __pthread_wait_for_restart_signal (self=0x43009be0) at
pthread.c:969
#2  0x404dc1ec in __new_sem_wait (sem=0x804d5c8) at restart.h:34
#3  0x402960e0 in waitSem (semId=0x804d5c8) at util.c:1126
#4  0x4027ae0f in dequeueAddress (notUsed=0x0) at address.c:425
#5  0x404d8c6f in pthread_start_thread (arg=0x43009be0) at manager.c:284

And select a specific frame via:

(gdb) select-frame 4

(gdb) help list
List specified function or line.
With no argument, lists ten more lines after or around previous listing.
"list -" lists the ten lines before a previous ten-line listing.
One argument specifies a line, and ten lines are listed around that line.
Two arguments with comma between specify starting and ending lines to list.
Lines can be specified in these ways:
  LINENUM, to list around that line in current file,
  FILE:LINENUM, to list around that line in that file,
  FUNCTION, to list around beginning of that function,
  FILE:FUNCTION, to distinguish among like-named static functions.
  *ADDRESS, to list around the line containing that address.
With two args if one is empty it stands for ten lines away from the other
arg.
(gdb) list address.c:425
420
421         while((myGlobals.addressQueueLen == 0)
422               && (myGlobals.capturePackets) /* Courtesy of Wies-Software
<wies@wiessoft.de> */
423               ) {
424     #ifdef USE_SEMAPHORES
425           waitSem(&myGlobals.queueAddressSem);
426     #else
427           waitCondvar(&myGlobals.queueAddressCondvar);
428     #endif
429           key_data.dptr = data_data.dptr = NULL;


(gdb) help print
Print value of expression EXP.
Variables accessible are those of the lexical environment of the selected
stack frame, plus all those whose scope is global or an entire file.

$NUM gets previous value number NUM.  $ and $$ are the last two values.
$$NUM refers to NUM'th value back from the last one.
Names starting with $ refer to registers (with the values they would have
if the program were to return to the stack frame now selected, restoring
all registers saved by frames farther in) or else to debugger
"convenience" variables (any such name not a known register).
Use assignment expressions to give values to convenience variables.

{TYPE}ADREXP refers to a datum of data type TYPE, located at address ADREXP.
@ is a binary operator for treating consecutive data objects
anywhere in memory as an array.  FOO@NUM gives an array whose first
element is FOO, whose second element is stored in the space following
where FOO is stored, etc.  FOO must be an expression whose value
resides in memory.

EXP may be preceded with /FMT, where FMT is a format letter
but no count or size letter (see "x" command).

(gdb) print key_data
$3 = {dptr = 0x0, dsize = 1076452514}
(gdb) print data_data
$4 = {dptr = 0x0, dsize = 4}


==========================================================================
==========================================================================
Original version Luca Deri, 1999-2001
Updated Burton M. Strauss III 2002, 2003, 2004, 2005

