TODO $Id: TODO 11866 2009-01-24 23:10:05Z fyodor $ -*-text-*-

o [Zenmap] Use a more efficient algorithm to update the display of Nmap normal
  output in live scans.
  zenmapGUI.NmapOutputViewer.NmapOutputViewer.refresh_output calls
  zenmapCore.NmapCommand.NmapCommand.get_output, which re-reads the
  entire output file (into memory) and then puts it in the text buffer
  if it has changed. So already we're storing the whole output twice in
  memory. When the text field changes, update_output_colors
  re-highlights the whole file.

o Update changelog to note recent changes

o Do final dev/test release

==STUFF WE WANT IN STABLE RELEASE ABOVE THIS LINE==

o The -V option to Nmap, in addition to reporting the version number,
  should give details on how Nmap was compiled and the environment it
  is running on.  This includes things like whether SSL is enabled,
  the platform string, versions of libraries it is linked to, and
  other stuff which is often useful in debugging problems.

o Process Nmap survey and send out results [Fyodor]

o Do new Nmap release with the stuff merged from SoC students and
  other new developments.
  - Need to have our SoC successes writeup for 2010 posted first.

o Write and post 2010 SoC Successes writeup [Fyodor]

o Make new SecTools.Org site with the 2010 survey results.

o A trivial change: we currently print some lines about NSE
  pre-scanning and post-scanning in verbose mode even when no such
  scripts are being run.  We should not print those in that case.  For
  example, nmap -A -v scanme.nmap.org gives me these superfluous lines:
  NSE: Script Pre-scanning.
  NSE: Starting runlevel 1 (of 2) scan.
  Initiating NSE at 12:23
  Completed NSE at 12:23, 0.00s elapsed
  NSE: Starting runlevel 2 (of 2) scan.
  NSE: Script scanning 64.13.134.52.
  NSE: Starting runlevel 1 (of 2) scan.
  Initiating NSE at 12:24
  Completed NSE at 12:24, 4.14s elapsed
  NSE: Starting runlevel 2 (of 2) scan.
  NSE: Script Post-scanning.
  NSE: Starting runlevel 1 (of 2) scan.
  NSE: Starting runlevel 2 (of 2) scan.

o Script review:
  - p2p-dropbox-listener http://seclists.org/nmap-dev/2010/q4/689
  - SCADA scripts http://seclists.org/nmap-dev/2010/q4/612
    http://seclists.org/nmap-dev/2010/q4/613
    http://seclists.org/nmap-dev/2010/q4/623
    http://seclists.org/nmap-dev/2010/q4/639
  - servicetags http://seclists.org/nmap-dev/2010/q4/691
  - Martin Swende patch to force script run
    http://seclists.org/nmap-dev/2010/q4/567

o We should document Ron's sample script
  (http://nmap.org/svn/docs/sample-script.nse) in docs/scripting.xml so
  that new script writers know about it.

o We should add fields to the service submitter
  (http://insecure.org/cgi-bin/submit.cgi?new-service) for the
  application name and version.

o Investigate ways to limit Winpcap privileges so that only
  administrative users or a certain accounts can sniff.  Maybe there
  is a solution people use for Wireshark or does it always cause this
  issue (allowing any user to sniff the network) when it is installed?
  - CACE says they will add a feature to do this.  See this thread:
  http://seclists.org/nmap-dev/2010/q3/826

o Add some content to https://secwiki.org and announce it.

o Create new default username list:
  http://seclists.org/nmap-dev/2010/q1/798
  o Could be a SoC Ncrack task, though should prove useful for Nmap
    too
  o We probably want to support several lists.  Like an admin/default
    list like "root", "admin", "administrator", "web", "user", "test",
    and also a general list which we obtain from spidering from
    emails, etc.

o Revive the Nmap Public Source License project (need to find an open
  source attorney to review it).  http://nmap.org/npsl/
  o Also take close look at Mozilla's license modernization project:
    http://mpl.mozilla.org/scope/

o Add raw packet IPv6 support, initially for SYN scan
 o After that can add UDP scan, and sometime OS detection (David did
   some research on what IPv6 OS detection might require).

o Further brainstorm and consider implementing more prerule/postrule
  scripts:
   o AS Number to IP ranges: http://seclists.org/nmap-dev/2010/q2/101
   o IPv6 Neighbor Discovery Protocol:
     http://en.wikipedia.org/wiki/Neighbor_Discovery_Protocol
   o DNS service discovery (Bonjour): http://en.wikipedia.org/wiki/Bonjour_%28software%29
   o Broadcast ping (could ping broadcast address and either report
     IPs+Mac addresses that it sees, or even add them to the scan queue
     if requested).
   o Netbios Name Service
   o DHCP broadcast requests
   o Postrules could be created which give final reports/statistics or
     other useful output.  Like a reverse-index, which shows all the open
     port numbers individually and the hosts which had that port open
     (e.g. so you can see all the ssh servers at once, etc.)
     Admittedly you can do that pretty easy with Zenmap instead.
   o We could have a prerule sniffer script which uses pcap to sniff
     traffic for some short configurable amount of time and then adds the
     discovered hosts to the target list.
   o We could have a script which takes traceroute results and adds them to the target list.
   o [Implemented] dns-zone-transfer
   o [Implemented, but a joke] http-california-plates

o [NCAT] Send one line at a time when --delay is in effect. This is
  cumbersome to do until Nsock supports buffered reading.

o [NCAT] Make the HTTP proxy support the chunked transfer encoding,
  then change it to be HTTP/1.1 and support pipelining.

o [NCAT] Drop privileges on startup.

o [NCAT] Work as a SOCKS4a/SOCKSv5 proxy.

o [NSE] Write a couple more MSRPC scripts inspired by sysinternals:
  o Windows system logs (like sysinternals' psloglist)
  o Services (like sysinternals' psservice)
  [Drazen]

o [NSE] Script writing contest (something to think about)

o [NSE] Consider using .idl files rather than manually coding all the
  MSRPC stuff.  The current idea, if we do this, is to have an
  application in nmap-private-dev which converts .idl files to LUA
  code for nmap/nselib. Consider adapting the pidl utility from Samba.

o [NSE] Consider a script (or modification to smb-check-vulns) to
  detect this MSRPC vulnerability:
  http://seclists.org/fulldisclosure/2010/Aug/122

o nmap.cgi web interface for Nmap
 - We're working on Rainmap hosted scanning system -- see /nmap-exp/rainmap
 - Should have "demo" mode that only allows users to scan their own addy

o Investigate and document how easy it is to drop Ncat.exe by itself
  on other systems and have it work.  We should also look into the
  dependencies of Nmap and Zenmap.  It may be instructive to look at
  "Portable Firefox"
  (http://portableapps.com/apps/internet/firefox_portable) which is
  built using open source technology from portableapps.com, or look at
  "The Network Toolkit" by Cace
  (http://www.cacetech.com/products/network_toolkit.html).  For Nmap
  and Nping, we may want to improve our Winpcap to load as a DLL
  without requiring installation.  There is a separate TODO item for that.

o [Zenmap] should actually parse and use script results. See
  http://seclists.org/nmap-dev/2010/q1/1108

o We should document an official way to compile/test refguide.xml so
  people can more easily test their changes to it.  This will probably
  involve moving legal-notices.xml into /nmap/docs, among other
  things.
  o Note that nping has its own /nmap/nping/docs/genmanpage.sh - we
  could look at how that could apply to Nmap.

o Nmap book work [placeholder]

o Implement a solution for people who want NIST CPE OS detection
  results (we'll save version detection for a 2nd phase).  Notes:
  David report on CPE for OS Detection: 
    http://seclists.org/nmap-dev/2010/q3/278
  David report on CPE for version detection: 
    http://seclists.org/nmap-dev/2010/q3/303
  Nessus has described their integration of CPE:
    http://blog.tenablesecurity.com/2010/05/common-platform-enumeration-cpe-with-nessus.html.
  Older messages about it:
    http://seclists.org/nmap-dev/2008/q4/627
    http://seclists.org/nmap-dev/2010/q2/788

o Make the nmap.header.tmpl wording a little more generic so it more
  clearly applies to Ncat, Zenmap, Nping, etc. Then use
  templatereplace.pl to apply those changes to the code. [Fyodor]

o Move Zenmap man page from nmap/docs/ to nmap/zenmap/docs to match
  the man page location for ncat and ndiff.
 o Don't break packaging/build system
 o Don't break the system for posting html to web site. 
 o Consider standardizing names for nping and ncrack man pages as well.
 [Fyodor]

o Consider an update feed system for Nmap which let's people obtain
  the latest Nmap data files, such as NSE scripts/libs, nmap-os-db,
  nmap-service-probes, etc.
  o Note that some scripts require updated compiled libraries.  We
    will need some sort of compatability system.
  o One approach is "svn up".  Note that Metasploit uses that approach
    even for Windows by shipping .svn directories and an svn executable
    with the Windows installer.  In taht case we might need to have a
    separate branch for each release that gets updated version/OS
    databases and scripts.
  o Another approach is a special feed system as is used by Nessus and
    OpenVAS.  OpenVAS uses a script wrapper around rsync, or an HTTP
    download if that fails.

o Investigate why and whether we need mswin32/pcap-include/pcap-int.h.
  This file is not included in the official WinPcap 4.1.1 developers'
  pack
  (http://www.winpcap.org/install/bin/WpdPack_4_1_1.zip). Presumably
  it covers internal functions and structures which we aren't really
  supposed to access it.  If we can get rid of it, that would be
  great.  If we need it, we should probably upgrade to the
  4.1.1. version (presumably from the Winpcap source code
  distribution). Right now it is included in tcpip.h,
  nsock/src/nsock_pcap.h, and nping/common_modified.cc: o David looked
  into it.  He says it isn't distributed with the WinPcap developer's
  pack. You have to extract it from the source file. He updated to the
  4.1.1 version.  He says The entire reason we need it is so we can
  peek at the definition of struct pcap, so we can access the
  pcap.adapter member on Windows.  In order to pass it to
  PacketSetReadTimeout. Usually struct pcap is an opaque type and you
  are only supposed to access it through a pcap_t *. Unfortunately I
  don't think there's an easy way to manipulate the timeouts in
  WInPcap like we do on other platforms. You can specify a timeout
  when you do pcap_open, but we like to set a timeout on every
  read. So we sort of sneak in and call PacketSetReadTimeout. In the
  code there's even a comment: "BUGBUG: This is cheating." libdnet
  also uses the Packet* functions, but in a more innocuous
  way. It doesn't access them through a struct pcap, so it
  doesn't need pcap-int.h.  David is going to test whether this makes
  any signficiant difference--we might be able to just remove the
  PcapSetReadTimeout().

o [Web] Add a page with the Nmap related videos we do have already

o [NSE] MSRPC - Improve domain support all around -- in particular,
  let the user give the domain in the format DOMAIN\username or
  username@DOMAIN anywhere that usernames are accepted. Suggested
  at http://seclists.org/nmap-dev/2010/q2/389

o [NSE] Combine similar MSRPC scripts, especially the "get info"
  stuff.  See this thread on combining
  (http://seclists.org/nmap-dev/2010/q1/1023).  This was suggested by
  Ron at http://seclists.org/nmap-dev/2010/q2/389.

o [Zenmap] Investigate getting new OS icon art. See
  http://seclists.org/nmap-dev/2010/q1/1090

o We should probably enhance scan stats--maybe we can add a full-scan
  completion time estimate? Some ideas here:
  http://seclists.org/nmap-dev/2010/q1/1007

o [NSE] Do some benchmarking of our brute.nse.  We should check the
  performance with different levels of thread parallelism.  Our
  initial results show that it isn't helping much for vnc-brute or for
  drda-brute (which is currently using the multi-thread feature
  directly rather than through brute.nse library).  We should figure
  out why the threads aren't helping more, and whether there is
  something we can do to fix it.  It would also be interesting to
  compare speed with Ncrack for services we have in common.

o We should offer partial results when a host
  timeouts. I (Fyodor) have been against this in the past, but maybe
  the value is sufficient to be worth the maintenance headaches.  Many
  users have asked for this.  If we do implement this, we may want to
  only print results for the COMPLETED phases (e.g. host discovery,
  port scanning, version detection, traceroute, NSE, etc.)  Trying to
  print partial results of a port scan or NSE or the like might be a
  pain.  And if we print some results for a host which timeouts, we
  should give a very clear warning that the results for that host are
  incomplete. As an example, here is someone who hacked Nmap source
  code to achieve this: http://seclists.org/pen-test/2010/Mar/108.
  o Another benefit would be that it would allow us to clean
    up/regularize the host output code. Right now there are I think
    three places where a host's final output can be printed. If,
    instead, that code just looked at what information was available and
    printed that out only, we could potentially isolate it in just one
    place.
  o This also might let us provide a feature for skipping the rest of
    an Nmap phase which is going too slowly (I think that has its own
    Nmap TODO item).

o [NSE] Consider a script which uses Nmap's detected OS and version
  detection information for open ports to print out _possible_ (unverified)
  vulnerabilities.  Of course it is better to have scripts which
  actually check for vulnerabilities, but we don't have comprehensive
  vuln detection yet, so this could still be quite useful to see what
  vulns _might_ exist on the software running on a remote machine.
  o Marc Ruef is working on a vulnscan.nse script which uses OSVDB to do
    this. See this thread: http://seclists.org/nmap-dev/2010/q2/527

o Consider providing an option which causes Nmap to scan ALL IP
  addresses returned for a given name.  So if "google.com" returns 4
  names, scan them all (right now we print them all but only scan
  the one which happens to be the first on the current list).  We then
  might want to make -A imply that option.  Here is a thread on the
  topic: http://seclists.org/nmap-dev/2010/q2/302
  - Note that we now have a script which does something similar
    this--resolveall.nse

o Start project to make Nmap a Featured Article on Wikipedia.
 - See http://seclists.org/nmap-dev/2010/q1/614

o Nmap should have a better way to handle XML script output.
 o We currently just stick the current script output text into an XML tag.

o Add Nmap web board/forum
 - First step is looking at the available software for this.

o [Zenmap] Consider a couple ideas from Norris Carden
  (http://seclists.org/nmap-dev/2010/q2/228):
 - remember last save and/or open location for new saves and/or opens
 - default save location option

o [Nsock] Consider adding server support to Nsock so it can accept
  multiple connections and multiplex the SD's, like it does for
  clients.  This could potentially be used by Ncat and Nping echo
  mode.  Currently Ncat server doesn't use Nsock at all, while Nping
  echo mode basically polls, repeating a loop of 1s in nsock_loop
  followed by a nonblocking accept().  Then Nping gives the SD's to
  Nsock to manage.

o [Nsock] Consider implementing a nsock_pcap_close() function or making
  nsp_delete() call pcap_close() when pcap IODs are used. Currently valgrind
  warns about a socket descriptor left opened (at least in Nping).
  ==10526==    at 0x62F77A7: socket (syscall-template.S:82)
  ==10526==    by 0x4E348A5: ??? (in /usr/lib/libpcap.so.1.0.0)
  ==10526==    by 0x4E36819: pcap_activate (in /usr/lib/libpcap.so.1.0.0)
  ==10526==    by 0x4E375FC: pcap_open_live (in /usr/lib/libpcap.so.1.0.0)
  ==10526==    by 0x4311A9: nsock_pcap_open (nsock_pcap.c:64)
  ==10526==    by 0x428078: ProbeMode::start() (ProbeMode.cc:329)

o Consider rethinking Nmap's -s* syntax for specifing scan types
  o Current problems with this -s syntax:
    o We already use like 20 of the 26 letters, so we end up with
      things like SCTP scan using -sY
    o Can make Nmap command lines hard to read, particularly given
      that we often need to improvise to find a letter which isn't
      taken.
    o Problematic for scan types -sI and -b which require arguments
    o Inconsistencies.  For example, -sC and -sV do script scan and
    version detection, respectively, and yet for OS detection we use
    -O.  Also, control flow (-sP, -sL) is used with -s, which further
    overloads the options.
  o Possible solution:
    o We are enabling -Pn and -sn as preferred notations for -PN and
      -sP which mean "no ping" and "no port scan".  Those match the
      already existing -n for "no DNS".  The problem with -sP is that it
      implies "ping only", when what it really should mean is "disable
      port scan" because you may want to do NSE, OS detection,
      traceroute, etc. still.
    o We might want to just give them normal option strings, so you
      could do --maimon instead of -sM, for example.  For extremely
      common options such as SYN scan, UDP scan, version detection, we
      could perhaps find good single letter options as an alias to the
      longer one.
    o Another idea is to use something like --scantype syn,udp,sctp,
      which is a lot longer for single-type scans, but shorter when
      you're combining mulitiple ones.  Doesn't allow for individual
      scan arguments easily.  I (Fyodor) think I prefer the idea above
      of just givem them top level arguments.
    o If we keep -s*, we could just give it one defined function, such
      as selecting port scan type, or control flow.
  o Obviously this will take some discussion/brainstorming on nmap-dev.

o Do -p- Internet UDP scans.

o [Ncat] This may sound ridiculous, but I'm starting to think that
  Ncat should offer a very simple built-in http server (e.g. for simply
  sharing files, etc.)  And maybe a simple client too.

o Scanning through proxies
 o Nmap should be able to scan through proxy servers, particularly now
   that we have an NSE script for detectiong open proxies and now that
   Ncat can act as proxy client or server.
 o Requirements:
   o Would be nice to be able to chain through multiple proxy servers of
     different types.
   o Would be nice to be able to spread the load amongst multiple
     proxies.
   o Should support port scanning, version detection, and NSE.  In
     other words, nsock should support proxies.
   o Support IPv4 and v6
   o Need to figure out how to get good performance.  Pool of
     connections to proxy or proxies for concurrency?  HTTP pipelining?
   o Support the different varieties of proxies: socks4, socks4a,
     socks5, HTTP GET (if possible), HTTP CONNECT.  Note that GET
     proxies present some challenges since the error messages may not
     be standard, etc.
   o Maybe auto-detect the proxy type so that Nmap can try the most
     efficient scanning method first?
   o I've been asked to support basic, ntlm, and digest authentication
     if possible.
 o Implementation ideas: 
   o There is a patch by Zoltan Panczel (http://nmap-dev.fw.hu) and it
     has been improved by Jacob Appelbaum in nmap-exp/ioerror/ .  This
     patch doesn't handle things like parallelization, but it may be a
     good proof of concept.
   o This might not be appropriate for ultra_scan ... perhaps would be
     better to write a general scanning engine for abusing
     applications for port scanning purposes.  This could handle
     scanning through proxies and the existing FTP bounce scan would
     also be ported to this engine (or, frankly, we could probably get
     away with removing FTP bounce).  rembrandt at jpberlin.de tells me
     that you can also do this with the "forwarding" commands on IMAP
     servers.  Whoever does this should probably start by reading the
     code for the main port scanning engine (ultra_scan()) and also
     the version detection code (service_scan()).  And the version
     detection paper at http://nmap.org/book/vscan.html.  If you
     understand all that, you may be ready for this project :).  This
     is important, because it is easy to do poorly.  The tough part is
     high performance and clean code which is general enough that all
     these different applications can be scanned through using the
     same basic engine.  You should run your ideas by nmap-dev in as
     much detail as possible before starting.

o [Ncat] Drop privileges once it has started up, bound the ports it
  needs to, etc.

o [Web] Consider adding training/introduction videos to the Nmap site
 o Would be great to have a (5 minute or less) promotional video
   introduction to each tool (Nmap, Zenmap, Ncat, Ndiff) on its web
   page.
 o They need to be good to be useful--the sort of the quality you see
   in Laura Chappell's Wireshark videos or James Messer's Nmap videos
   or Irongeek's videos (http://www.irongeek.com).
 o Besides the promotional videos, users would probably enjoy more
   in-depth video instructions (e.g. covering the Nmap Network
   Scanning topics).
 o Here's an example product page with lots of videos (we may not go
   that far): http://www.splunk.com/product

o The Zenmap translation system
  (http://nmap.org/book/zenmap-lang.html) has been pretty successful
  so far.  We should consider doing the same for Nmap.  After all, we
  already have the reference guide in 16 languages at
  http://nmap.org/docs.html.  We should definitely try to use the same
  translation methods for Zenmap as we do for Nmap.  In fact, maybe we
  can create a combined PO file Nmap, Zenmap, Ncat, and Ndiff so that
  they can all be translated and maintained together. Something to
  consider: calling setlocale can change the behavior of functions like
  isalpha. Locale-dependent functions need to be checked for security
  risks.

o Consider changing Nsock so that it is able to take advantage of more
  modern interfaces to dealing with large sockets, rather than just
  select.  Perhaps we should look at poll(), Windows completion ports,
  and some of the advanced Linux APIs.  Select() limits us to
  descriptors no higher than FD_SETSIZE, and it may not performa all
  that well.  We should do some benchmarking and decide on the
  interface to use for each platform. May want to take a look at
  libevent (http://www.monkey.org/~provos/libevent/) for inspiration.
  The libevent home page has some interesting benchmark graphs too.
  [Josh implemented poll as a SoC student, but it had problems with
  Nsock's architecture.  O(1) lookups were becoming O(n) because of
  the nature of the data structures. It was slower in his benchmarks.
  Nsock would have change from a model of "loop over the event list,
  and check to see if the fd for each event is set," to one of "loop
  over the fd list, and see if there is a corresponding event for
  each. It is the "see if the fd is set" operation that's O(1) with
  select (it's FD_ISSET) and O(n) with poll (it's a traversal of a
  linked list).]

o [NSE] Consider whether we should include some sort of NSE debugger.  Or we
  could include something simpler.  For example, Nmap now provides a
  traceback (with sufficient debugging/verbosity) when a script ends
  in error.  For some inspiration/ideas, look at Diman's NSE
  debugger (http://seclists.org/nmap-dev/2008/q1/0228.html).

o [NSE] We may want to consider a better exception handling method --
  one which doesn't require wrapping every I/O line in its own try
  function call.  David says "Lua has an internal "exception handling"
  mechanism based on a function called pcall, which is implemented
  with setjmp/longjmp.  You can wrap a function call in it and the
  function will return there whenever there's an unhandled error.
  Something based on that would be better [than the current system], I
  think."

o [NSE] Consider whether we need script.db for performance reasons at
  all or should just read through all the scripts and parse on the fly.
  See: [http://seclists.org/nmap-dev/2009/q2/0221.html]

o [NSE] Support routing http requests through proxies.

o [NSE] http improvements
 o Spidering library+scripts?  How should the spider store the results
   and make them available to other scripts?  How do we limit
   bandwidth consumption and total amount of data stored?  Might want
   to look at enumeration script at
   http://seclists.org/nmap-dev/2009/q1/0889.html
 o URL grinder checks for existence of applications in common/default
   paths. Scanning http paths to see if they exist is in some ways
   similar to scanning to see which ports are open.
 o Cookie suppport?  Might be useful for spidering sites which use it
   for authentication/authorization/personalization.
 o HTTP persistant connections/keepalive? May make
   spidering/grinding/auth cracking more efficient
 o Pipelining? May make spidering/grinding/auth cracking more efficient

o Consider offering a way to link Winpcap DLLs so that they start the
  service as needed rather than requiring explicitly installing
  Winpcap and having it start upon system boot.  CACE has offered such
  a thing for many years as WinPcap Pro
  (http://www.cacetech.com/products/winpcap_pro.html).  If we change
  WinPcap in this way, we'd presumably want to also change the symbol
  names as is done in WinPcap Pro.  And it would mean that we have to
  build our Winpcap binaries ourselves (including 64-bit).  We might
  even have to sign our drivers for 64-bit Windows.

o [NSE] BasicHTML/XML parser?  For example, Sven Klemm wrote a script
  which uses libxml2: http://seclists.org/nmap-dev/2008/q3/0462.html.
  And here is one by Duart Silva using Expat:
  http://seclists.org/nmap-dev/2009/q3/1093.

o [NSE] Would be great if NSE scripts could be made to NOT
  run as root if they don't have to.


o [NSE] Consider how we compare to the Nessus Web Application Attack
  scripts
  (http://blog.tenablesecurity.com/2009/06/enhanced-web-application-attacks-added-to-nessus.html).
  [Joao making a list of web scripts which we might find useful,
  Fyodor asking HD moore for permission to use http enum dir list]

o [NSE] Security Review
 o Consider what, if any, vulnerabilities or security risks NSE has
   with respect to buffer overflows, format string bugs, any other
   maliciously formatted responses from target systems, etc.  Maybe
   address the known risk of malicious scripts too.
 o Consider that NSE runs scripts as root

o Figure out and document (in at least the Ncat user's guide) the best
  way to use Ncat for chaining through proxies.  One option is this
  sort of thing:
  ncat -l localhost 1234 --sh-exec "ncat --proxy A.A.A.A B.B.B.B"
  ncat --proxy localhost:1234 C.C.C.C
  If you had two proxies A.A.A.A and B.B.B.B, connecting to C.C.C.C.
  With another listener/--sh-exec pair for each additional proxy.
  But perhaps we can make it easier by adding it to the syntax.

o Look into whether we should loosen/change the global congestion
  control system to address possible cases of one target host with many
  dropped packets slowing down the whole group.  See
  http://seclists.org/nmap-dev/2008/q1/0096.html .
  * Related possibility: Fix --nogcc to gracefully handle ping scans.
    Right now it seems to go WAY TOO FAST (e.g. several thousand
    packets per second on my DSL line).
  * [12/22/09] David says: It still is in one case that I've
    documented on my wiki. I had an idea to fix it, but on testing it
    it didn't work. The idea was to treat the global congestion limit
    differently. Instead of dropping it down to the minimum level on a
    drop as is done currently, I thought about only dropping it by the
    amount that the individual host limit drops. For example, if a
    host had a drop and its limit fell from 25 to 1, then the global
    limit would change (if it was at 100 to begin with) to 76, not all
    the way down to 2 or whatever it is.  The idea being that the
    global limit is most important at the beginning of a scan, when
    there's no information to set host limits, and every host wants to
    send all its first probes at once. See
    http://www.bamsoftware.com/wiki/Nmap/PerformanceNotesArchive2#global-cc. I
    am convinced, though, that some sort of global control is
    necessary. There's a reason that a web browser limits the number
    of connections it will make, and doesn't try to download every
    image file at once and count on the fairness of TCP to sort it
    out.

o Make Zenmap settings get upgraded when the Zenmap executable is
  upgraded. The per-user configuration files such as scan_profile.usp
  and zenmap.conf are never overwritten once installed by Zenmap, so
  changes and fixes to those files don't reach anyone who has
  installed Zenmap already. This is most noticeable with changes to
  profiles and highlight definitions are notably affected. This fix
  may involve hard-coding settings that are not normally configured by
  users (like highlighting) or updating the per-user files at startup
  (only those parts that haven't been changed by the user).

o libnmap organization for UNIX and Windows
  o Then change Nmap and Zenmap to simply call this library
  o It is interesting to look at: http://www.gnupg.org/gpgme.html

o Improve the "run Zenmap as root" menu item to work on distributions
  without su-to-root.  We might even want to improve Zenmap so that it
  itself does not have to run as root, and just executes Nmap that
  way.  Rather than not showing Zenmap as root on the Menu of
  non-working systems, it might be better to have it but let it give
  an error message (and then, perhaps, run as nonroot) so that users
  of those distributions are more likely to contribute a fix.  We also
  might want to look at how the distributions themselves package Zenmap.

o Deal with UDP retransmission for version detection (I think I
  should just do a second run of all probes for UDP if it fails to
  match anything).  The advantage there is that no retransmissions are
  neccessary if the service is found.  Then again, per-probe
  retransmission would let us redo the most likely probes (the one(s)
  that match the port number) quickly.  Lost packets should probably
  affect ideal_parallelism.

o Nmaprc-related - Create a system to store Nmap defaults/preferences
  in an nmaprc file.
  o nmaprc should be in ~/.nmap on UNIX
  o On Windows, we may need a registry key to find the .nmaprc
  o Obtain Nmap data directory information from nmaprc at runtime rather than
    compiled in -- among other advantages this is needed to make
    relocateable rpm.
  o Make RPM relocatable (requires somehow avoiding storing paths in the
    binary)
  o Perhaps Lua could be used as the format?
  o .nmaprc for keeping defaults, etc.
    o Nmaprc infrastructure, hook to new timing variables
    o Nmaprc man page
    o Default timing mode
    o Default NSE arguments, such as user agent
    o Maybe Default source IP (-S) argument
    o should be a way to specify your own .nmaprc
    o Maybe lets you add a directory and template for saving all
     scans. 

o Search for nmap on google news, on google web, and add appropriate
  links to press page and the like.

o Make version detection and NSE timing system more dynamic so that
  the concurrency can change based on network conditions/ability.
  After all, beefy systems on fast connections should be able to handle
  far more parallel connections than slower systems.

o Get new Zenmap logo
 o consider putting back on top-right of command constructor wizard
 (there used to be umit logo there).
 o Maybe that can be done after the release by soliciting ideas.

o Create or collect some great ./configure ascii art.

o Add randomizer to configure script so that a random ASCII art from
  docs/leet-nmap-ascii-art*.txt is printed.  I think I'll start naming
  them leet-nmap-ascii-art-submittername.txt.

o Add general regression unit testing system to Nmap
 o David has created a great system for Ncat which could serve as a
   model.

o Provide an option to send a comment in scan packet data for target
  network.  Examples: --comment "Scan conducted by Marc Reis from
  SecOps, extension 2147" or --comment "pH33r my l3eT
  s|&lt;iLLz!  I'll 0wN UR b0x!"
  o Note, this shouldn't be implemented yet.

o Consider implementing RPC scan with ultra_scan or something else.
  Right now it is the only program using pos_scan.  On the other hand,
  I'm not sure TCP rpc scanning is appropriate for ultra_scan.

o Look at all the pcap functions, there are some like
  pcap_findalldevs() which could be quite useful.  There are mails to
  the Nmap list relating to suggested improvements --
  http://seclists.org/lists/nmap-dev/2004/Apr-Jun/0024.html .
  Actually I do indirectly use that for Windows. I wonder if they work
  for UNIX?

o perhaps each 'match' line in nmap-service-probes should have a
  maximum lines, bytes, and/or time by which a response should be
  available.  Once that much time (or many bytes or lines) have passed,
  that match can be considered 'failed' and ignored in subsequent runs.
  Once all matches are considered failed, that probe is done.  This
  could be a useful optimization and is arguably better than the less
  granular 'totalwaitms'.  Or I could just have a simple function that
  looks at whether a given regex could possibly match something
  starting with the received data (not too hard since almost all of
  the current regexes are anchored).  But before doing this, I should
  look long and hard at how many of the probes have every match
  capable of doing this.  In particular, many of the softmatch lines
  don't offer many chars anchored at the front.

o Add detection of duplicate machines via IP.ID technique.
   Maybe I should use uptime timestamps too.  Oh, and MAC addresses
   too.  Our SSH host key script is useful for this as well.

o Separate nbase into its own Windows library in the same way as Andy did
   with iphlpapi .

o Look into iplog ( http://ojnk.sourceforge.net/ ) -z option which is 
  supposed to fool OS detection.

o More security auditing of Nmap code (it never hurts to do more proactive
  security auditing).

o Nmap / Nmap-hackers FAQ

o random tip database

DONE:

o If Nping is compiled w/o SSL support, and the user specifies an
  encryption key, it should fail and insist they use --no-crypto
  rather than ignoring the key and omitting crypto.  Otherwise the
  user might think they're getting encryption when they're not.  David
  found this problem in the server, and we also should check how the
  client behaves.

o [Ncat] Make --exec work in conjunction with --proxy. The --proxy
  code path skips the --exec code. See
  http://seclists.org/nmap-dev/2010/q4/604 and the test "--exec
  through proxy" in ncat-test.pl.

o Decide what to do about Nmap static binaries failing to work on new
  Fedora releases (and others?).  See these threads:
  http://seclists.org/nmap-dev/2011/q1/46 and
  http://seclists.org/nmap-dev/2010/q1/308
  o We ended up dynamically linking system libs in the RPM rather than
    statically linking them.  We still statically link things like lua,
    pcre, ssl, etc.

o Fix our mac builds so that they contain SSL support again (5.35DC1
  did, but TEST1 and TEST2 didn't for some reason.

o Add our broadcast discovery scripts to a "broadcast" category (they
  should generally just be in "broadcast" and (assuming they are safe)
  "safe", and not normal "discovery".  Update scripting.xml to note
  this new category too.

o The latest IANA services file
  (http://www.iana.org/assignments/port-numbers) has many identified
  services which are still "unknown" in our files because ours is
  based on a much older version of that file.  We should probably take
  that file and add names and comments to our nmap-services-all where
  they are "unknown" in our file.  An example of such a port is 3872,
  oem-agent.

o Script review:
  - patch for ftp-proftpd-backdoor
    http://seclists.org/nmap-dev/2010/q4/678
  - patch for hddtemp-info http://seclists.org/nmap-dev/2010/q4/676

o We should probably update our Windows build systems to use Python
  2.7.  As of 11/8, it looks like all our dependency libraries are
  available for 2.7:
  o David upgraded and it worked, though Rob found a potential problem
  and added vcredist 2008.  Fyodor will test on the official Win7 Nmap
  build system.
  PyGTK: 2.22.0 IS available for 2.7
  PyCairo: 1.8.10 IS available for 2.7
  PyGObject: 2.26.0 IS available for 2.7
  Py2exe: 0.6.9 IS available for 2.7

o Do service/version detection submission integration (last done in
  April)

o Do os detection submission integration (last done in April)

o Script review:
  - modbus-enum http://seclists.org/nmap-dev/2010/q4/489

o Create Nmap wiki
 o Decide on domain name
 o Include insecure Chrome
 o Decide on wiki software, probably just use mediawiki
 o install it on a Linode, probably Web

o [NSE] Web application fingerprinting script.  Would be great to be
  able to take a URL and determine things like "this is Joomla" or
  "this is Plone" or "Mediawiki" or whatever. Rather than hard code
  regular expressions or other tests in a script, it should use a
  signature file like Nmap OS and version detection do.  Might work in
  combination with URL grinder to check for applications at
  default/common locations. See also a script that does favicon
  scanning TODO item.
  - http-enum pretty much does this now.

o Update our distribution build systems and documentation to use
  Visual C++ 2010 Express rather than the 2008 version.  See
  http://www.microsoft.com/express/Windows/

o Dependency licensing issues (OpenSSL, Python, GTK+, etc.)
 o Almost done!  We just have some file renaming/organizing left to do.
 o We should do an audit to ensure that we are in complete compliance for the
 licenses of all the software we ship in any of our downloads, as some
 licenses have special clauses for things like including their
 license/copyright file, mentioning them in our documentation, etc.
 And of course we want to credit them properly even where the license
 doesn't require it.  We should probably make a list of these in our
 docs/ directory along with any special information/requirements of
 their license.  And maybe we should put the current licenses in a
 subdir too.  In particular, these come to mind:
 o libpcre
 o lua
 o OpenSSL
 o libpcap
 o GTK+/Glib/ATK/Pango/PyGTK (Win/Mac versions of Zenmap link to
   PyGTK)
 o SQLite
 o Python (Win/Mac versions of Zenmap link to Python)
 o X.org libraries (Mac version links to them)
 o libdnet

o Small NSEDoc bug:
  http://nmap.org/nsedoc/scripts/dns-zone-transfer.html contains 'id
  \222\173' near the bottom.  This is presumably due to misparsing this
  line from the script: local req_id = '\222\173'.  Given that we don't
  use IDs any more, maybe we can just get rid of the functionality.

o [NSE] We should probably enable broadcast scripts to work better by
  (initial thoughts):
  o Done and merged by David!
  1) Change NSE to always set nsp_setbroadcast() on new sockets
  2) Change nsock to create real sockets at time of nsi_new so you can
     bind to them.
  See this thread (only some of the messages involve broadcast
  support): http://seclists.org/nmap-dev/2010/q3/357

o [NSE] Review scripts:
   o Hostmap (Ange Gutek) - http://seclists.org/nmap-dev/2010/q3/159

o Post BH/Defcon Nmap videos

o Let Nsock log to stderr, so its messages don't get mixed up with the
  output stream when Ncat is run with -vvv.
  http://seclists.org/nmap-dev/2010/q3/113

o [NSE] Our http-brute should probably support form POST method rather
  than just GET because some forms require that.

o Nping needs to call nsp_delete so that its socket descriptors are
  not left behind.

o [Zenmap] Add a button to select script files from the filesystem. 

o [Zenmap] Show help for individual script arguments in the Help pane,
  not for all arguments at once.

o Upgrade our Windows OpenSSL binaries from version 0.9.8j to the
  newest version (1.0.0a as of Aug 12, 2010).

o Since Libdnet files (such as ltmain.sh) are apparently only used by
  libdnet (they used to be used by shared library NSE C scripts), we
  should move them to the libdnet directory.
  o Turned out to be a pain.  See
    http://seclists.org/nmap-dev/2010/q3/733

o [Zenmap] Consider a memory usage audit. This thread includes a claim
  that a 4,094 host scan can take up 800MB+ of memory in Zenmap:
  http://seclists.org/nmap-dev/2010/q1/1127
  The reporter mentioned Guppy/Heapy to debug memory use:
  http://guppy-pe.sourceforge.net/
  http://www.pkgcore.org/trac/pkgcore/doc/dev-notes/heapy.rst.  Many
  Nmap survey respondants complained about this too.
  Note: Fyodor has a 50MB scan log file named ms-vscan.xml which
  demonstrates this problem.  When trying to load the file, Zenmap
  grows to 1150MB of RAM, pegs the CPU usage at 100% for many
  minutes or maybe hours (I forgot about it, but woke up the next day
  to find that it had started, was then using 2.4GB of RAM.  The
  hosts/services functionality seemed to work, although it would take
  a minute or so to switch from say "ftp" port to view "ssh" ports.

o [NSE] Maybe we should create a script which checks once a day
  whether similar tools (Metasploit, Nessus, OpenVAS, etc.) have any
  new modules, and then mails out a list of them with the description
  fields.  The mail could go to just interested parties, or maybe
  nmap-dev.  This may help prevent important vulnerabilities from
  falling through the cracks.  Perhaps we would include new NSEs in
  there too, especially if we open it up as a public list.

o Now that NSE has more script phases (prerule, postrule, hostrule,
  portrule, and versionrule soon to come), the NSEDoc should specify
  which phases a script belongs to.

o Consider implementing a nsock_pcap_close() function or making
  nsp_delete() call pcap_close() when pcap IODs are used. Currently valgrind
  warns about a socket descriptor left opened (at least in Nping).
  See http://seclists.org/nmap-dev/2010/q3/305.
  o It turns out that the pcap descriptors are being closed properly,
    but Nping isn't calling nsp_delete.

o [NSE] High speed brute force HTTP authentication.  Possibly POST and
   GET/HEAD brute force cracking. [done except for form POST, adding
   separate TODO item for that]

o [NSE] Review scripts:
   o New brute, vnc, and svn scripts by Patrik.  This guy is a coding
     machine :).  http://seclists.org/nmap-dev/2010/q3/111
   o rmi-dumpregistry by Martin
     Swende. http://seclists.org/nmap-dev/2010/q2/904
   o path-mtu.nse - http://seclists.org/nmap-dev/2010/q3/222
   o 15 more from Patrik :).  http://seclists.org/nmap-dev/2010/q3/284

o [NSE] Consider modifying our brute force scripts to take advantage
  of the new NSE multiple-thread parallelism features.
 - We've done this with db2-brute, but the DB may have been a
   bottleneck there, so we should probably do more testing after
   modifying another script for this sort of parallel cracking.

o Look into implementing security technologies such as DEP and ASLR on
  Windows: http://seclists.org/nmap-dev/2010/q3/12.

o Ncat and Nmap should probably support SSL Server Name Indication
  (SNI). See this thread: http://seclists.org/nmap-dev/2010/q3/112.
  We need this to talk to web servers which share one SSL IP and port
  because we need to ask for the right SSL key.

o [NSE] In the same way as our -brute scripts limit their runtime by
  default, I think qscan should be less intense by default.  For
  example, perhaps it could run by default on no more than 8 open
  ports, plus up to 1 closed port.  Right now it does things like
  running on 65,000+ closed ports and bloats scan time (and output).
  Of course there could (probably should) still be options to enable
  more intense qscanning.

o [Web] We should see if we can easily put the Insecure chrome around
  Apache directory listings and 404 pages (e.g. http://nmap.org/dist/
  and http://nmap.org/404).  I think we may have had this working
  before the move to Linode, so maybe check conf/httpd.conf.syn.

o Do a serious analysis if and how we should use the NIST CPE standard
  (http://cpe.mitre.org/) for OS detection and (maybe in a different
  phase) version detection results.  One thing to note is that they
  may not have entries for many vendors we have.  For example, one
  person told me they couldn't find SonicWall or D-Link in the CPE
  dictionary.  Here are some
  discussions threads on adding CPE to Nmap:
  http://seclists.org/nmap-dev/2008/q4/627 and
  http://seclists.org/nmap-dev/2010/q2/788.
  Nessus has described their integration of CPE at
  http://blog.tenablesecurity.com/2010/05/common-platform-enumeration-cpe-with-nessus.html.

o [NSE] Create NSE scripts to scan for and/or exploit these VXWorks issues:
  http://blog.metasploit.com/2010/08/vxworks-vulnerabilities.html [Ron
  may be able to do this.  Or others are welcome to take a shot at it.]

o The -g (set source port) option doesn't seem to be working (at least
  in Fyodor's quick tests) for version detection or connect() scan,
  and apparently doesn't work for NSE either.  We should fix this
  where we can, and document the limitation in the refguide where it
  is impractical. Also see http://seclists.org/nmap-dev/2010/q2/576.

o [Zenmap] script selection interface for deciding which NSE scripts to
  run.  Ideally it would have a great, intuitive UI, the smarts to
  know the scripts/categories available, display NSEdoc info, and even
  know what arguments each can take.

o Review http-xst (Eduardo Garcia Melia) -
  http://seclists.org/nmap-dev/2010/q3/159

o [NSE] Investigate sslv2.nse falsely reporting SSLv2 as being
  supported.
  http://seclists.org/nmap-dev/2010/q2/754

o [NSE] The NSEDoc for some scripts includes large "Functions"
  sections which aren't really useful to script users.  For example,
  see http://nmap.org/nsedoc/scripts/snmp-interfaces.html.  Perhaps we
  should hide these behind an expander like "Developer documentation
  (show)".  I don't think we need to do this for libraries, since
  developers are the primary audience for those documents.
  o Talked to David.  We should just remove the function entries.

o We should add a shortport.http or similar function because numerous
  services use this protocol and many of our scripts already try to
  detect http in their portrule in inconsistent ways.

o [NSE] Maybe we should create a class of scripts which only run one
  time per scan, similar to auxiliary modules in Metasploit. We
  already have script classes which run once per port and once per
  host. For example, the once-per-scan ("network script"?) class might
  be useful for broadcast LAN scripts (Ron Bowes, who suggested this
  (http://seclists.org/nmap-dev/2010/q1/883) offered to write a
  NetBIOS and DHCP broadcast script). Another idea would be an AS to
  IP ranges script, as discussed in this thread
  http://seclists.org/nmap-dev/2010/q2/101 [Could be a good SoC
  infrastructure project]
  o David notes: "I regret saying this before I say it, because I'm
    imagining implementation difficulties, we should think about
    having such auxiliary scripts be able to do things like host
    discovery, and then let the following phases work on the list it
    discovers."

o Analyze what sort of work would likely be required for Nmap to
  support OS detection over IPv6 to a target.
  o Would probably start with a way to send raw IPv6 packets
    o There is a raw IPv6 patch here:
      http://seclists.org/nmap-dev/2008/q1/458
    o Also it looks like Nping may be doing this already.
  o Then we need to figure out if we can use our current DB and
    techniques, or if we'd likely thave to have an IPv6-specific
    DB. [David]

o July Nmap releases (at least a beta version, and maybe a stable
  too).  Last release was 5.30BETA1 on March 29

o Add this patch for compilation on OpenSolaris.
  http://blogs.sun.com/sdaven/entry/nmap_5_35dc1_compile_on

o Now that we've put the ndiff, ncat, and nping man pages under the
  scope of the book (e.g. http://nmap.org/book/ncat-man.html), we need
  to add a redirect from the old locations and also update our links.

o Make sure the long output lines in Nping's man page are OK for the book.
  See r18829 and r18864.

o Update "History and Future of Nmap"
  (http://nmap.org/book/history-future.html) to include all the news
  since September 2008. [Fyodor]

o Fix Win7 networking issue reported by Luis which seems to have been
  triggered by r17542. See this thread:
  http://seclists.org/nmap-dev/2010/q3/40

o Upgrade to WinPcap 4.1.2 - Rob has a patch - See this thread:
  http://seclists.org/nmap-dev/2010/q3/18

o [NSE] Review UnrealIRCd backdoor detection script
  http://seclists.org/nmap-dev/2010/q2/854

o [Zenmap] Investigate segfault on some installs of OS X 10.6.3:
  http://seclists.org/nmap-dev/2010/q2/587
  o David rebuilt with MacPorts 1.9.1 rather than 1.8.2 and the
    problem went away.

o [Zenmap] Investigate failure to start on some installations of OS X
  10.6.3.
  [ We think one may just not have waited long enough as he said it
  started working, and another case (the 587) seems to be a
  segfault--we added a new task for that ]
  http://seclists.org/nmap-dev/2010/q2/587
  http://seclists.org/nmap-dev/2010/q2/859 (He responded to David
  privately and said that it was not an I7 processor.)
  Nmap seems to be having problems too:
  http://seclists.org/nmap-dev/2010/q2/747

o [NSE] Review Gutek's PHP version disclosure script.
  http://seclists.org/nmap-dev/2010/q2/569

o Fix the IPv6 name resolution problem described in this thread:
  http://seclists.org/nmap-dev/2010/q2/787

o [NSE] Review Gutek's libopie detection/DOS script.
  http://seclists.org/nmap-dev/2010/q2/635

o [NSE] Review Gutek's web server directory traversal script.
  http://seclists.org/nmap-dev/2010/q2/595
  - It became modifications to http-passwd

o [NSE] Review dns-cache-snoop.nse from Eugene Alexeev.
  http://seclists.org/nmap-dev/2010/q2/195
  Better attachment at: http://seclists.org/nmap-dev/2010/q2/200
  Need to decide on a domain list: http://seclists.org/nmap-dev/2010/q2/199

o Fix bug where multiple targets with the same IP can end up in a
  hostgroup and cause port scanning and probably OS detection to
  misbehave.  An example is "nmap -F scanme2.nmap.org
  scanme3.nmap.org". See this thread for details:
  http://seclists.org/nmap-dev/2010/q2/322

o Need to fix our current win32.zip distribution so that .svn files
  aren't included (currently they are in nselib/data).  Will probably
  be a simple adjustment to mswin32/Makefile.

o Make Zenmap splash screen

o [NSE] Add one of, or combine, ntp-peers and ntp-monlist.
  http://seclists.org/nmap-dev/2010/q2/190
  http://seclists.org/nmap-dev/2010/q2/191
  
o [NSE] Reorganize nselib to allow libraries in subdirectories.
  Currently, to avoid expanding the number top-level libraries, code
  that is only used by one library is built into that library's file,
  even if it is logically separate. For example, the mongodb library
  contains a BSON-parsing library. Instead, that library could go in
  mongodb/bson.lua. The msrpc and smb libraries could potentially be
  broken up in this way.
  UPDATE: We decided not to do this for now, given complications in
  nsedoc, packaging, etc. to support the new hierarchy.  Instead, we
  can use prefixes like we do with scripts (e.g. mongodb-bson.lua,
  msrpc-types.lua).

o Add a configure option to our libpcap which enables an older Linux
  packet capture system (David's noring patch).  This is needed in
  some cases for 32-bit static binaries to work on 64-bit Linux
  systems.  Note that it is unneccessary if both the build system and
  the target system use Linux 2.6.27, as that has an architecture
  independent tpacket_hdr (called tpacket2_hdr).  [Added by David as
  --disable-packet-ring]

o Test Jay Fink's UDP payload prototype.
  http://seclists.org/nmap-dev/2010/q1/168
  [ tested, improved, merged by David]

o Resolve Ncat broadcast support issue (see this thread:
  http://seclists.org/nmap-dev/2010/q2/422).

o [NSE] Review and test the DB2 library and
  scripts. http://seclists.org/nmap-dev/2010/q2/395 (but updated
  versions may be available).

o Move nmap/docs/TODO into its own todo directory (probably nmap/todo)
  and then encourage maintainers of /status/ TODOs and any other TODOs
  to migrate theirs there.  Unlike the status directory, /nmap/todo
  would be readible by anyone. [Fyodor]

o Nmap should at least print (and maybe scan) all IP addresses for
  hostnames specified on the command line.  We will start with just
  printing all the addresses. Here is a thread on the topic:
  http://seclists.org/nmap-dev/2010/q2/302
  [David made it do the printing, adding a different task related to
  scanning them all]

o Integrate new service detection fingerprint submissions (we have
  more than 730 since Dec. 17, 2009.

o [Ncrack] Use our new password lists (now used by NSE) for Ncrack as
  well.  Ncrack can probably handle a larger list than NSE uses.

o Consider MSRPC ideas from Ron--we might want to add some as TODO
  tasks: http://seclists.org/nmap-dev/2010/q2/389

o Fix XML inconsistency described at
  http://seclists.org/nmap-dev/2010/q2/326

o Integrate new OS fingerprints (we have more than 1,300 since
  November 10, 2009).

o Finish selecting GSoC 2010 projects

o Upgrade libpcap to the new 1.1.1 version.

o Improve the NSI installer by adding command-line options for unsetting
   each of these GUI checkboxes individually (particularly useful for
   silent mode):
   LangString DESC_SecCore ${LANG_ENGLISH} "Installs Nmap executable, NSE scripts and Visual C++ 2008 runtime components"
   LangString DESC_SecRegisterPath ${LANG_ENGLISH} "Registers Nmap path to System path so you can execute it from any directory"
   LangString DESC_SecWinPcap ${LANG_ENGLISH} "Installs WinPcap 4.1 (required for most Nmap scans unless it is already installed)"
   LangString DESC_SecPerfRegistryMods ${LANG_ENGLISH} "Modifies Windows registry values to improve TCP connect scan performance.  Recommended."
   LangString DESC_SecZenmap ${LANG_ENGLISH} "Installs Zenmap, the official Nmap graphical user interface.  Recommended."
   LangString DESC_SecNcat ${LANG_ENGLISH} "Installs Ncat, Nmap's Netcat replacement."
   LangString DESC_SecNdiff ${LANG_ENGLISH} "Installs Ndiff, a tool for comparing Nmap XML files."
   LangString DESC_SecNping ${LANG_ENGLISH} "Installs Nping, a packet generation tool."

o We should have a standard function which takes time arguments in the
  same format as Nmap does (e.g. 60s, 1m, etc.) and the scripts which
  take time arguments should be modified to use it. David suggests
  this here: http://seclists.org/nmap-dev/2010/q2/35. We are also
  going to update the normal Nmap timing functions to take seconds by
  default, as described here: http://seclists.org/nmap-dev/2010/q2/159

o Nmap should probably always produce a well-formed XML file, even if
  it exits with a fatal() error.  In that case, the error should be
  included in the XML.  Right now, for example, if the network is
  down, the XML output will just stop (no closing tags) and Nmap will
  print something to STDERR like:
  nexthost: failed to determine route to 9.48.184.164
  QUITTING!

o Get @output sections for the last remaining scripts w/o them:
  [WARN] script auth-spoof missing @output
  [WARN] script db2-das-info missing @output
  [WARN] script db2-info missing @output
  [WARN] script http-passwd missing @output
  [WARN] script iax2-version missing @output
  [WARN] script ms-sql-config missing @output
  [WARN] script ms-sql-query missing @output
  [WARN] script oracle-sid-brute missing @output
  [WARN] script pop3-brute missing @output
  [WARN] script pptp-version missing @output
  [WARN] script skypev2-version missing @output

o [Zenmap] Maybe it should sort IPs in an octet-aware way. And maybe
  you should be able to sort by IP address (perhaps that should be the
  default).  Current plan is to just sort by IP by default, and maybe
  we'll offer other sort techniques later if desired.  See
  http://seclists.org/nmap-dev/2010/q2/27 [possible SoC student task]

o Brainstorm for GSoC 2010 ideas and fill out the org application by
  Friday 3/12 4PM PST.
  o NSE scripts
   o Maybe a whole SoC role for http scripts
     o Maybe look at other web app scanners for some inspiration
      (including w3af - http://w3af.sourceforge.net/)
   o Maybe a non-http developer too
  o NSE infrastructure manager
  o Ncrack
  o Nping
  o Mobile Devices? N900, iPhone, Android
  o Zenmap developer 
    o Must have solid user interface design experience
    o Zenmap script selector (subset of a Zenmap or NSE SoC role)
  o Feature Creepers/Bug fixers

o Review IDS detection scripts from Joao Correa.
  http://seclists.org/nmap-dev/2010/q1/814

o Review mssql library and scripts from Patrik Karlsson.
  http://seclists.org/nmap-dev/2010/q1/1000 (files)
  http://seclists.org/nmap-dev/2010/q1/1014 (sample output)

o Review DNS fuzzer script from Michael Pattrick.
  http://seclists.org/nmap-dev/2010/q1/1005

o Our nsedoc generator should probably give a warning if a script is
  missing any important fields. @output comes to mind.  @usage can be
  nice too, though we could consider auto-generating that for trivial
  scripts.

o [NSE] Consider pros and cons of splitting information retrieval
  scripts into a bunch of small single-purpose script vs. one larger
  argument-controlled script.  See
  http://seclists.org/nmap-dev/2010/q1/1023
  [we ended up combining three of the ms-sql scripts. If we combine
  future scripts, we need to remember to add them to the deprecation
  list in the Makefile]

o Remove --interactive.  It was broken for a long time and nobody
  seemed to notice, and we put a call out on nmap-dev for
  --interactive users and didn't get any good reasons to keep it.  We
  should kill it to remove the code complexity it adds and to avoid
  the documentation complexity of people having to read and learn
  about a feature they are unlikely to ever use.

o Zenmanp should perhaps be able to print Nmap output on a Printer (if
  not too much of a pain to implement.)

o Review afp-serverinfo.nse from Andrew Orr.
  http://seclists.org/nmap-dev/2010/q1/470 Just waiting on some bug fixes:
  http://seclists.org/nmap-dev/2010/q1/665

o Test 64-bit pcap installer (e.g. remove old version and install new)
  before next release, as we've applied a change from Rob which works on
  his system (http://seclists.org/nmap-dev/2010/q1/796).

o [NSE] Improve username/password library (the database files
  themselves).  We don't have very good lists at the moment.  Maybe
  work in combination with Ncrack dev.
  o Now there are some even better lists available (f.e. RockYou)--see
    this thread: http://seclists.org/nmap-dev/2010/q1/764
  o We've improved the ncrack files--we should probably either use
    those for NSE or use a subset of them.
  o perhaps from Solar Designer. (he sent us permission)
  o perhaps add phpbb hack data (there is at least a list of 28,635
    passwords in phpbb_users.sql, and possibly more in other files.

o [Nping] Should take the version number 0.[nmap version], such as
  0.5.22TEST

o Review rpc.lua, nfs-showmount.nse, nfs-get-stats.nse, and
  nfs-get-dirlist.nse from Patrik Karlsson.
  http://seclists.org/nmap-dev/2010/q1/270

o [NSE] Look into moving packet module to C for better performance
  [Patrick]
  o Removing this one because it is stale (has been here for many
    months with no action seen), but it is something we can consider
    if/when there is a desire to implement it.  A key is probably to
    measure current performance and see if it is a material problem.

o Maybe the Nmap ASCII art should come after make rather than
  configure?
  - We decided it would probably be annoying for developers to see it
    every time they 'make'.

o Review snmpenum.nse from William Njuguna.
  http://seclists.org/nmap-dev/2009/q4/721
  http://seclists.org/nmap-dev/2010/q1/656
  o Dropping for now unless original author or someone else picks it
  up and fixes the bugs.

o Add smtp-enum-users from Duarte Silva if testing is favorable.
  http://seclists.org/nmap-dev/2010/q1/699

o After the new -sn and -Pn options (added to SVN around 7/20, just
  after the 5.00 release) have been around long enough to be in most
  people's copy of Nmap (e.g. in all the versions we distribute from
  download page (stable+dev)) for at least a few months, we'll document
  these as the preferred version rather than -sP and -PN.  These match
  -n, and the main problem with -sP is that we now use it more for
  "disable portscan" than ping only.  For example, you can also use
  NSE, traceroute, etc. [David]

o Nmap currently selects routes based on the first matching one it
  finds.  But it should really take the most specific route instead.
  So it should:
  1) Keep searching the routing table for the most specific match, and
  2) Use a stable sort (not qsort) so that routes with identical
     netmasks aren't rearranged.
  For more, see http://seclists.org/nmap-dev/2010/q1/685

o Review pgsql-brute.nse from Patrik Karlsson.
  http://seclists.org/nmap-dev/2010/q1/455

o psexec missing (need to download yourself now) nmap_services.exe
  output issue: "The function where this is detected returns a value
  that is passed to stdnse.format_output. format_output takes a
  parameter to decide whether it's displaying an error message, but it
  is hard-coded to only display error messages with debugging >= 1. So
  options are to change format_output and make it more flexible, or
  somehow decouple the sensing of nmap_service.exe from the normal
  output channel of the script."

o Website: Create shared directory in svn, which will contain
  directories shared between the Insecure.org network of sites
  (e.g. templates, error, css).  Then sites such as sectools,
  nmap.org, insecure.org can just check that out via externals
  declaration (or, I suppose, symlink).  CSS directives will then use
  /shared/css/insecdb.css etc. ).

o Add CouchDB and JSON scripts once the JSON library is finished.
  http://seclists.org/nmap-dev/2010/q1/641

o Review NSE raw IP from Kris Katterjohn.
  http://seclists.org/nmap-dev/2010/q1/559

o Review sslv3-enum.nse from Mak Kolybabi.
  http://seclists.org/nmap-dev/2010/q1/563

o [NSE] Consider LDAP library and scripts from Patrik Karlsson.
  http://seclists.org/nmap-dev/2010/q1/70 [all merged, except David is
  still reviewing ldap-search]

o More potential improvements to http-methods:
   http://seclists.org/nmap-dev/2010/q1/630 and
   http://seclists.org/nmap-dev/2010/q1/640

o Remove smtp-open-relay.nse sometime after 9/24/09 if nobody adopts it (see
  http://seclists.org/nmap-dev/2009/q3/0986.html). [It got fixed up
  and we kept it.]

o The -v and -d arguments should take the same syntax.  Right now you
  use -vvv vs. -d3. We should probably just make either approach work
  with either of them.

o Zenmap should be able to export normal Nmap output

o Integrate Nping.

o [NSE] Consider the http-methods script from Bernd Stroessenreuther.
  http://seclists.org/nmap-dev/2010/q1/76. [integrated, but David is
  making some improvements].

o The Nmap web page is beginning to show its age.  Ah, who am I
  kidding, it was showing its age 5 years ago :).  It could do with an
  upgrade to XHTML+CSS.  It could also do with a whole redesign, but I
  think that can be done as a second step after converting to
  XHTML+CSS with roughly the same look.  Though adding a few more
  modern touches (like hover interaction on the menu bar) wouldn't
  hurt.  This is a moderatly big project, which will involve: o
  Designing the new XHTML+CSS to look similar to the current HTML
  pages, but be extensible enough that it can be redesigned in the
  (near) future by mostly just changing the CSS and graphics.  
    o Converting the existing Nmap pages to the new XHTML format.
      This will likely include using open source programs and likely
      modifying them or creating your own scripts to help with the
      process.  To apply for this task, you need to have some web
      development experience and an example XHTML+CSS web page you
      have created online.
    o We decided not to worry about XHTML for now, and we're
      integrating CSS in piece by piece -- we already have the section
      headers, left sidebar links. etc.
    o Should not use SSI like the current pages -- should do all its
      magic through CSS. That way it will work on seclists too (which
      can't do SSI for security reasons).
    o Maybe alpha transparency for menus, gradiants, curves, etc.  But
      the main goal isn't flashiness.

o Seclists.org should maybe be fixed so that it doesn't strip quoted
  text for its summaries from the IP list because that list consists
  almost entirely of forwarded material which is being stripped.  Look
  at the summaries at http://seclists.org/interesting-people/.

o Web site HTML improvements
  - Maybe start with nmap.org.
    - Find and fix HTML validation problems, bad links.  I'm not sure
      what tool is best for this.
  - Then do the same with seclists.org, insecure.org, sectools.org
  - The icon on the top-left of the screen should be for (and link
    to) the root URL of current site.  e.g. seclists.org,
    sectools.org, nmap.org rather than always insecure.org.

o [NSE] Consider SNMP scripts from Patrik Karlsson.
  http://seclists.org/nmap-dev/2010/q1/162
  http://seclists.org/nmap-dev/2010/q1/174
  http://seclists.org/nmap-dev/2010/q1/178

o Deal with AV false positive issue RE nmap_services.exe:
 - For now, David is going to apply Ron's patch which removes this,
   but David will make it print output in verbose mode rather than
   debug and maybe make it a little less verbose. LT plan is for Ron
   to encrypt it with OpenSSL.

o Web site improvements
    - Update to use CSS, at least for header bars
      - Also, if it is easy to give the header bars rounded corners,
        we should probably do so.  But if it is hard, it isn't
        important enough to matter.
    - The Nmap.Org navigation table should have a background and more
      subtle lines, like we use for our calendars now.
    - The first item (table) in featured news has slightly more
      left/right margin than the later ones on Firefox 3.5.6, and with
      IE8 it doesn't extend as far when you make the page really wide.
      Plus the images on the right are problematic (extend through the
      border below them) when you make the window too wide on IE8.
      Having a slight margin on the left/right of entries would
      actually be a bit nice.  And it would be nice if it only took a
      simple tag or two, controlled by CSS rather than pasting in a
      whole table with font tags and the like for each entry.

o [Ncat] Test, review, and (if appropriate) merge Venkat's HTTP Digest
  proxy authentication patch.  See
  http://seclists.org/nmap-dev/2009/q3/773. [David]

o [NSE] Look at new DB2 script by Tom
  Sellers. http://seclists.org/nmap-dev/2009/q4/659

o [NSE] Consider MongoDB scripts and libraries from Martin Holst Swende.
  http://seclists.org/nmap-dev/2010/q1/177

o [NSE] Document Patrick's worker thread patch in scripting.xml (see
  http://seclists.org/nmap-dev/2009/q4/294,
  http://nmap.org/nsedoc/lib/stdnse.html#new_thread,
  http://nmap.org/nsedoc/lib/nmap.html#condvar) [Patrick]

o Make Nmap 5.21 bugfix-only release

o [NSE] Consider afp-showmount script from Patrik Karlsson.
  http://seclists.org/nmap-dev/2010/q1/97
  [merged to trunk]

o [NSE] Review DNS-SD script from Patrik Karlsson.
  http://seclists.org/nmap-dev/2010/q1/87
  [merged to trunk]

o [NSE] Consider MySQL scripts from Patrik Karlsson.
  http://seclists.org/nmap-dev/2010/q1/163
  [merged to trunk]

o [NSE] Consider DAAP script from Patrik Karlsson.
  http://seclists.org/nmap-dev/2010/q1/164
  [merged to trunk]

o NSEDoc left sidebar should include a link to
  http://nmap.org/book/nse.html below "Index".

o Consider enhancing the new OS Assist system to handle version
  detection too. [We decided not to do this as David noted that Doug's
  serviceunwrap.lisp does pretty much everything he needs.]

o [NSE] HTTP header parsing is not very robust, and is duplicated in a
  lot of places. For example, it's legal to have header fields like
Content-type:\r\n
___text/html\r\n
(with spaces in place of _, but http.lua won't parse such a header
correctly. In other words you can extend them to any number of lines
as long as each line after the first begins with whitespace. [David]

o Investigate issue with our Pcap and Wireshark x64, as described in
  this thread: http://seclists.org/nmap-dev/2009/q4/557 [Rob]
  [Taking this off the list until/unless we get more reports]

o Decide what to do about Windows 7/Vista and starting NPF.  See this
  thread: http://seclists.org/nmap-dev/2010/q1/20

o [NSE] We should do a favicon survey like the one Brandon did for
  /favicon.ico files but which uses the favicons specified by the HTML
  files rather than just that exact location.  For example, insecure.org
  sites include in the headers:
  <link REL="SHORTCUT ICON" HREF="http://images.insecure.org/images/tiny-eyeicon.png" TYPE="image/png">
  Then we should update our favicon database to include the top ones,
  and we should also improve our favicon script so that it either
  omits checking /favicon.ico if the HTML-specified one exists, or it
  should just download, interpret, and display info for both (right
  now it seems to give prority to the wrong one: /favicon.ico).


o [Ncat] Add SSL support for --exec so you can use SSL to talk to your
  remote shell, etc.  See this thread:
  http://seclists.org/nmap-dev/2009/q4/255, particularly the
  implementation sketch at http://seclists.org/nmap-dev/2009/q4/268 [Venkat,David]

o Look at new Kerberos script from Patrik Karlsson.
  http://seclists.org/nmap-dev/2009/q4/715 .  [We decided not to merge
  this one since its usefulness turned out to be limited on Windows and
  very limited on any other platform. ]

o Add feature to http library to let user set the user agent to be
  used. The NSEDoc for this feature should probably tell what our
  current default user agent is ("Mozilla/5.0 (compatible; Nmap
  Scripting Engine; http://nmap.org/book/nse.html") [David]

o On our NSEDoc pages (e.g. http://nmap.org/nsedoc/), perhaps the link
  text for scripts should not include the ".nse".  Basides saving
  horizontal space, this may improve the sorting so that the likes of
  "citrix-enum-apps" comes before "citrix-enum-apps-xml".  Also, we can
  probably get away with reducing the width of the NSEDoc left-column,
  especially if ".nse" is removed.

o [NSE] Patrick's script dependency patch:
  http://seclists.org/nmap-dev/2009/q4/295
  o I'm not sure if he has gone through and actually set appropriate
    dependencies (and removed runlevels) yet

o Integrate latest version detection submissions and corrections.
  This was last done based on submissions until February 9, 2009.

o Release 5.10BETA2

o Add --evil to set the RFC3514 evil bit.
  ftp://ftp.rfc-editor.org/in-notes/rfc3514.txt
  o We're not going to add this right now.

o Talk to Libpcap folks about incorporating (at least some of) my
  changes from libpcap/NMAP_MODIFICATIONS. [marking as done since the
  upstream-appropriate changes are pretty minor now that we've
  upgraded to 1.0]

o Nping -- like hping3 but uses Nmap infrastructure and to a
  large degree the same command-line options as Nmap.
  [We now have an alpha version at http://nmap.org/nping/]

o Further investigate SCTP functionality, as some people reported
  problems (see this thread:
  http://seclists.org/nmap-dev/2009/q2/0669.html)

o [NSE] NFS query script for checking exports, etc.? [Patrik Karlsson]

o [NSE] Attempt to reproduce and fix a deadlock reported by Brandon
  when he does large-scale scanning with a new favicon script with
  hostgroups as small as 8,192 (he hasn't seen it with 4096
  hostgroups). Could be a bug in internal NSE socket lock. Probably
  not specific to the favicon script, but that is how Brandon
  reproduces it. At the hang, stack trace is usually the threads stuck
  in socket_lock function, sometimes lookup_cache mutex in http
  library. David guesses that it's threads being garbage-collected
  from the socket lock table. The only thing that can wake up a thread
  waiting on a socket lock is if a thread that holds a lock is removed
  from the table. But the table has weak keys, meaning that a thread
  can be garbage collected and it will be automatically removed from
  the table by the Lua runtime. Then there is no event that can wake
  up a thread waiting for a lock. [David and Patrick made some commits
  at end of November meant to resolve this, and we haven't seen the
  problem since, so we're marking it as done for now].

o Look into reducing Nmap memory consumption
  o UDP scans with -p- and large hostgroups are a particularly large
    offender.  See if there is a way to prevent them from eating up
    gigs of RAM. See the message "Port memory bloat" at
    http://seclists.org/nmap-dev/2009/q3/0926.html for a patch that
    reduces Port memory use by about 50%. 
  o One idea David has been considering is a way to represent filtered
    ports (or whatever the default state is) without creating a Port
    object for each one.
  [David]

o Fix assertion failure with certain --exclude arguments (see
  http://seclists.org/nmap-dev/2009/q4/276). [David]

o Many people may have stale (since removed/renamed) scripts in their
  Nmap scripts directory because our 'make install' does not remove
  them and so they remain and can cause problems (like running twice
  after being renamed).  We should probably add a line to our 'make
  install' which removes the scripts/lib names we have previously
  used.  We're doing this rather than blowing away the old directory
  just in case someone has custom scripts/libs there (though that is
  still a bad idea). [David]

o Update the CHANGELOG for new 5.10BETA1
  release. [Fyodor]

o Make the new Nmap 5.10BETA1 release

o Ndiff man page should be built from XML source whenever a release is
  done, as ncat/zenmap/nmap man pages are. [Fyodor]

o We should package the rendered Nroff man page translations (e.g. all
  16 languages) in the tarball to make it easier for distributors to
  package them.  For example, see
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=358336.  Including
  the translations would add 2.5MB to the (currently 28MB)
  uncompressed tarball and about 800KB to the (currently 9MB) bz2
  compressed tarball. [Fyodor]

o The Nmap 5.00 tarball contains:
  -rw-r--r-- fyodor/fyodor 122943 2009-06-24 14:35 nmap-5.00/docs/scripting.xml
  -rw-r--r-- fyodor/fyodor    151 2009-06-24 14:35 nmap-5.00/docs/nmap-usage.xml
  -rw-r--r-- fyodor/fyodor    604 2009-06-24 14:35 nmap-5.00/docs/nmap-man-enclosure.xml
  -rw-r--r-- fyodor/fyodor  76918 2009-06-24 14:35 nmap-5.00/docs/nmap-install.xml
  -rw-r--r-- fyodor/fyodor  10179 2009-06-24 14:35 nmap-5.00/docs/legal-notices.xml
  If we're going to include the XML source files, we should include
  refguide too.  But rather than add that, we should probably take
  these out.  After all, people can easily grab them from svn or our
  new http svn gateway if desired.  So no need to bloat the tarball
  with these files which aren't installed. [We're going to take the
  XML source files out of the tarball] [Fyodor]

o Consider converting this file to emacs org-mode
  (http://orgmode.org/) format. [Fyodor]
  o That format is still plain text and can be read/edited by vi
    users, etc.
  [Considered, but I don't think I'll change right now]

o Windows 7 RTM Nmap testing (With particular attention to 64-bit and
  our pcap installer). [Fyodor]

o We should print host latency (when available) in the XML output, as
  suggested at http://seclists.org/nmap-dev/2009/q4/215.
  docs/nmap.dtd will have to be modified accordingly, and you might
  even consider adding support to docs/nmap.xsl.

o Integrate latest OS fingerprint submissions and corrections. This
  was last done based on submissions up to May 8, 2009.

o Potential OS X 10.6 problems.  There are two issues reported by the
  same user which may be related:
  http://seclists.org/nmap-dev/2009/q3/0936.html,
  http://seclists.org/nmap-dev/2009/q3/0996.html.  One is that Nmap
  hangs doing nothing and needs to be killed with Ctrl-C, and the
  other is that it dies after printing "Initiating UDP Scan".  Another
  reported the same problem at
  http://seclists.org/nmap-dev/2009/q3/0990.html, where it dies after
  the first ARP request is sent.  But Brandon has run Nmap on 10.6
  without problems.  It is a bit of a mystery. [David] [Resolution:
  Apple fixed the problems in 10.6.2; For users who have 10.6 and
  10.6.1, the versions David builds on 10.5 will still work for them
  because they are 32-bit binaries rather than 64.  Users who build
  Nmap on 10.6 or 10.6.1 should compile with -m32 or update to 10.6.2]

o [NSE] Patrick's worker thread patch:
  http://seclists.org/nmap-dev/2009/q4/294

o Investigate get_rpc_results error (infinite loop) reported by Lionel
  Cons. See these threads: http://seclists.org/nmap-dev/2009/q4/24,
  http://seclists.org/nmap-dev/2009/q4/120

o Upgrade to latest version of NSIS on Nmap Win build system [Fyodor].

o Standardize on a proper file header for the Zenmap source code. [David]
 o For now, David is going to augment the templatereplacement system
   to insert the normal nmap.header.tmpl, but change the comment format
   to work with Python, and then replace the current Zenmap headers
   with that.

o We may want to look into if/how we support IPv6 nameservers.  Here
  is a bug report from someone having a problem with them:
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=539244 [Ankur]

o Once all the man page languages are in the Nmap tarball, we should
  update our install system to install them in the appropriate place.
  We'll want to integrate this with configure so users can decide which
  languages they want.  See http://seclists.org/nmap-dev/2009/q4/249.

o Resolve allow_ipid_match issue which can cause some malformed
  replies to be ignored when we might be able to still use them.  See
  this thread: http://seclists.org/nmap-dev/2009/q2/665 [David]

o Fix Zenmap 'make install' TypeError issue
  (http://seclists.org/nmap-dev/2009/q4/225). [David]

o Fix a bug in which Nmap can wrongly associate responses to SYN and
  ACK host discovery probes.  [David]
  For example:
  # nmap -sP -PS80 -PA80 australia.gov.au --packet-trace -d2
  SENT (0.0760s) TCP 192.168.0.21:60182 > 152.91.126.70:80 S ttl=43 id=13466 iplen=44  seq=4046449223 win=4096 <mss 1460>
  SENT (0.0770s) TCP 192.168.0.21:60182 > 152.91.126.70:80 A ttl=48 id=39976 iplen=40  seq=4046449223 win=1024 ack=921915001
  RCVD (0.3020s) TCP 152.91.126.70:80 > 192.168.0.21:60182 SA ttl=53 id=0 iplen=44  seq=3924706636 win=5840 ack=4046449224 <mss 1380>
  We got a TCP ping packet back from 152.91.126.70 port 80 (trynum = 0)
  ultrascan_host_probe_update called for machine 152.91.126.70 state UNKNOWN -> HOST_UP (trynum 0 time: 226875) Changing ping technique for 152.91.126.70 to tcp to port 80; flags: A
 In the example above, Nmap wrongly uses ACK as the preferred ping technique, when it should be SYN. [David]
 o we're thinking about ways to encode the information better.  Right
  now we have pingseq and tryno, but we may want to just move to a
  single probe ID and then we can look up any other information in
  structures attached to that ID in memory when we get the response.
 o A related problem, which we hope the fix for this will also
  resolve, is that replies can currently match any probe whose tryno
  is less than or equal to the tryno encoded in the reply.
   o However, "fixing" this problem has been shown in the past to
     cause accuracy problems.  See
     http://seclists.org/nmap-dev/2009/q1/387.  We should figure out
     whether we can still reproduce that and, if so, what is going on
     before "fixing" this issue.

o Add PJL (Printer Job Language) probes to
  nmap-service-probes. Brandon wrote some in
  http://seclists.org/nmap-dev/2009/q1/0560.html. Test them to see if
  they cause anything to be printed out (on paper) with printers that
  don't support PJL. If not, then remove the JetDirect ports from the
  default exclude list. The script pjl-ready-message.nse also uses
  PJL.  We have concerns about the safety of this probe given
  http://seclists.org/nmap-dev/2009/q4/61, but it still is probably
  better to have the probe in there than not, as long as we continue
  blocking the ports by default with the Exclude directive.
  [We put in the probes, but are keeping the Exclude directives
  because the probes still seem a bit dangerous]

o [NSE] in_chksum in packet.lua doesn't work with an odd number of
  bytes.  Also make it more efficient.

o Add --confdir option to Zenmap. See
  http://seclists.org/nmap-dev/2009/q1/92 [David]

o Update our Winpcap from 4.0.2 to 4.1.1
  (http://seclists.org/nmap-dev/2009/q4/128).  This is a bit complex
  because we have our own installer.  See
  http://nmap.org/svn/mswin32/winpcap/Upgrading-Instructions.txt.

o Change Nmap to not show the "Host not scanned" lines in list scan

o Change Nmap to show latency in "host is up" lines even w/o verbose
  mode.

o Update our included Libpcap from 0.9.7 to 1.0.0
  (http://www.tcpdump.org/) [David]

o Improve Nmap output to show the forward DNS name when specified on
  command line as well as rDNS where appropriate.  We're also going to
  reorganize output to enable some other improvements as well.  See
  the proposal at http://seclists.org/nmap-dev/2009/q3/814, and that
  whole thread which starts at
  http://seclists.org/nmap-dev/2009/q3/805 [David].

o [Zenmap] Solve some unusual utf8 Zenmap crashes reported in the
  crash reporter. David has fixed some of them so far, but there are a
  few more remaining that may be related. [David]

o Change Nsock to give an error if you try to FD_SET a fd larger than
  FD_SETSIZE. [Brandon]
  o Some research from David:
     We have help off on this change because of Windows portability
     problems. The Windows fd_set works differently than the Unix
     fd_set. In Unix, FD_SETSIZE (which is typically 1024) is both the
     maximum number of file descriptors that can be in the set and one
     greater than the greatest file descriptor number that can be
     set. In other words, we want to bail out whenever someone tries
     to FD_SET file descriptor 1060, for example. But on Windows it's
     different: FD_SETSIZE is only 64, but any file descriptor
     numbers, no matter how great, may be stored in the set. Windows
     socket descriptors are typically greater than 1023, but you can
     only have 64 of them in the set at once.

     So the fix on Unix would be
     --- nsock/src/nsock_core.c	(revision 15214)
     +++ nsock/src/nsock_core.c	(working copy)
     @@ -97,6 +97,7 @@
      do { \
        assert((count) >= 0); \
        (count)++; \
     +  assert((sd) < FD_SETSIZE); \
        FD_SET((sd), (fdset)); \
        (max_sd) = MAX((max_sd), (sd)); \
        return 1; \
     @@ -107,6 +108,7 @@
        assert((count) > 0); \
        (count)--; \
        if ((count) == 0) { \
     +    assert((sd) < FD_SETSIZE); \
          FD_CLR((sd), (fdset)); \
          assert((iod)->events_pending > 0); \
          if ((iod)->events_pending == 1 && (max_sd) == (sd)) \

     But that doesn't work on Windows (I just tried it) because even
     the smallest socket descriptor is bigger than FD_SETSIZE, 64.
     Really we're trying to accomplish two different things on the two
     platforms: On Unix we must not store a file descriptor greater
     than 1023, no matter how many or how few other descriptors have
     been set. On Windows we must not set more than 64 descriptors at
     a time, no matter what their descriptor number happens to be.

o Add a way in NSE to set socket source addresses and port numbers.
  See this thread: http://seclists.org/nmap-dev/2009/q3/821.  Some
  potential solutions are discussed later in the thread.

o [Ncat] Fix --max-conns on Windows so that it only counts concurrent
  connections and not long-dead ones.  See this thread
  (http://seclists.org/nmap-dev/2009/q3/1017.html) and particularly this
  message (http://seclists.org/nmap-dev/2009/q3/1032.html) for
  details.  Venkat has a patch for David to review and potentially merge.

o [Ncat] Fix 100% CPU usage with ncat -l --send-only.  See this
  thread: http://seclists.org/nmap-dev/2009/q2/797 and continues
  further at http://seclists.org/nmap-dev/2009/q3/99.  This message is
  key: http://seclists.org/nmap-dev/2009/q3/308 [David]

o [Seclists] There is currently some extra vertical space after the
  first post of a thread in the thread index (example:
  http://seclists.org/nmap-dev/2009/q4/index.html).

o [NSE] Decide which scripts belong to the "safe" category (we now have 20
  which aren't either safe or intrusive), then remove the intrusive
  category since people can now specify "not safe".  See
  http://seclists.org/nmap-dev/2009/q3/1091.html and that whole
  thread. [Fyodor]
  [ OK, see http://seclists.org/nmap-dev/2009/q4/0002.html]

o [NSE] Fix http pipelining.  Responses are being split on anything
  that looks like HTTP/1.X which doesn't come at the beginning of a
  line, and doesn't work when a line like that happens to legitimately
  come in a body.  Joao has an nmap-exp branch which resolves this
  issue, though David found some bugs in that and sent some hard test
  cases. [Joao]

o Fix traceroute performance/algorithms.  It is terribly bad in some
  cases.  For example, this traceroute scan took 36 minutes against a
  single host(!): http://seclists.org/nmap-dev/2009/q3/0425.html .  We
  don't need to go up to hop 50 in such cases (maybe some heuristic
  like "at least go to hop 15, and stop after 5 unresolved in a row).
  And more importantly, there is no reason each hop should take 40s to
  timeout.  It should probably use timeout variables like we use in
  port scanning.  And it should parallelize as much as possible.  Even
  if parallel resolution means we went a little further than we had to
  in incrementing the TTL, and we go to hop 15 when host is at 12
  that's no big deal (of course we would only report up to hop 12 in
  the output).  Once we do this, we should put back the ability to
  make --traceroute work even when we haven't found a probe which
  elicits a response from the target. (that feature was added in July,
  but we'll probably take it out until we can fix
  performance). [David]

o Fix four Nmap bugs discovered by Ankur and analyzed a bit by
  David. [Ankur]

o [NSE] Consider HTTP request caching.

o [NSE] Finish (or write new) favicon fingerprinting script. See
  http://seclists.org/nmap-dev/2008/q4/0583.html .  May need to do
  some more scanning and increase the DB size a bit.  May or may not
  want to later combine this as part of a larger webapp fingerprinting
  script.

o [Zenmap] When the inventory is changed, the current host/service selection is
  forgotten and the Ports / Hosts tab is switched to hosts mode. It should
  remember your current selection and not change the view. [David/SoC]

o Device categorization improvements
  o Examine Nmap's device categorization in nmap-os-deb and
    nmap-service-probes.  Decide if some small categories which have
    never really took off should be consolidated, or whether others
    should be split off.  For example, maybe there are some groups in
    'specialized' or other misc. categories which are now large enough
    to split off.  Personally, I wouldn't give anything its own
    category unless there are at least half a dozen of them and no
    other category really fits them well.  We should use a combined
    system for nmap-os-db and nmap-service-probes.
  o Add a classification sect1 to os-detection.xml
    (http://nmap.org/book/osdetect.html) to cover how Nmap handles OS
    classification.  It should include a list with descriptions of
    each device type recognized by Nmap.  Version-detection.xml should
    reference (link to) it in the approprate place.
    [Doug has done some initial work on this.  For example, see
     nmap/docs/device-types.txt] [David]

o Consider what new UDP payloads we might want to add.  David has many
  ideas at: http://seclists.org/nmap-dev/2009/q3/0290.html

o For traceroute we should give some indication that the RTT is in ms.
  Changing the column header to maybe "RTT MS" or "RTT (MS)" would
  probably do the trick or we could append "ms" to each value.
  [David]

o OS fingerprint should probably specify somewhow when DS=1 if it's
  because target->directlyConnected is true, or because it sent the
  distance probe and calculated a distance of 1.  The second situation
  should never happen, but often David strongly suspects that it is the
  case.

o --traceroute should probably set currenths->distance because right
  now, I do an -O scan against scanme.nmap.org, and it does not figure
  out the distance.  So the fingerprint shows no distance element and
  Nmap doesn't print "Network Distance" in the results line.  That may
  be OK (Nmap probably isn't receiving the probe response needed for
  this, and maybe doesn't want to print the TG), but even when I do
  --traceroute I get no distance printed.  Yet Nmap clearly knows the
  distance since the traceroute shows all the hops up to and including
  the target (scanme.nmap.org).

o Figure out best favicon to use for Nmap and related web sites
  [David]

o [Ncat] David says: "After you get EOF on stdin with --send-only, the
  program hangs on until the idle timeout expires instead of terminating
  immediately.  I had a fix for it but it involved deleting events in
  the Nsock queue and it caused an assertion failure in Nmap so I backed
  it out. I have a less intrusive solution." [David]

o We should update our config.{sub,guess} files.  This Debian bug
  #542079 requests that we do so:
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=542079.  We last
  updated on 3/15/08 and in that case we used versions from
  http://cvs.savannah.gnu.org/viewvc/config/?root=config.  That may or
  may not be the best place to get them now (e.g. perhaps there has
  been a recent official release). [David]

o Look a bit more at default version detection timing.  Particularly
  deciding the number of probes to run in parallel. [ We increased
  that a bit on 8/18/09]

o [Ncat] Right now our -i (idle timeout) causes Ncat to quit if EITHER
  reading or writing is idle for the given amount of time.  But it is
  really only idle if BOTH reading AND writing are idle for the
  period.  We should make the code work that way.

o Add scripting.xml documentation on strict.lua and the avoidance of
  global vars in libraries. See
  http://seclists.org/nmap-dev/2009/q3/0169.html. Probably a new
  section just above "Adding C Modules to "Nselib", such as "Writing
  Your Own Library" or somesuch. [Patrick]

o Update nsedoc to refer to 'libraries' rather than 'modules'.  This
  affects the front page (which calls them 'Libraries' on left sidebar
  and 'Modules' on the list of right, and affects the url (we should
  change /modules/ to /lib/ and then have Fyodor add a redirect for
  people still using old URLs) and the title of the module pages like
  http://nmap.org/nsedoc/modules/base64.html. [Patrick]

o [Ncat] Prefix Ncat stderr messages with "Ncat: " to make it clear
  that they are coming from Ncat and not the remote server (or typed in
  by user). [David/SoC]

o [NSE] Optimize NSE Performance--e.g. measure the current performance and
  see what can be improved in terms of scheduling scan threads,
  determining how many to run concurrently, looking at CPU load items,
  etc. [David/Patrick]

o Increase version scan concurrency based on Patrick's performance
  testing.  We decided to go to 20 for timing_level 3, 30 for 4, and 50
  for 5.

o [NSE] Consider POST/HEAD support. See
  http://seclists.org/nmap-dev/2009/q1/0889.html.
  o Implemented:  http://seclists.org/nmap-dev/2009/q3/0074.html
  o Joao going to check in very soon soon.

o [NSE] Consider Rob Nicholls http-enum script for incorporation:
  http://seclists.org/nmap-dev/2009/q1/0889.html
  [Joao tested w/his HEAD support, is going to check this in]

o Consider the open proxy scripts more carefully
 - How should we test whether the proxy attempt was successful?  Right
   now we look for a google-specific Server header after trying to
   reach http://www.google.com through the proxy.  Maybe we should let
   users specify their own pattern if they specify their own URL.
  [ Joao is going to check it in today (7/28)]

o I should add code to Nmap to bail if sizeof(char) isn't 1.
  Otherwise there could be security risks if it is not one on any
  platforms.  [ Actually, we think C standard requires this and we've
  not heard of any system where sizeof(char) isn't 1.  So removing
  this item.]

o [Zenmap] More complete implementation of ZenmapCommandLine/profile
  editor improvement ideas. See
  http://www.bamsoftware.com/wiki/Nmap/ZenmapCommandLine. [David]

o [Ncat] Think about whether we should offer "-q secs" (quit after EOF
  + delay of secs) and/or -k (set SO_KEEPALIVE on socket) (or maybe
  that should be set by default).  Anyway, these were suggested here:
  http://lwn.net/Articles/341706/ [We're going to fix -i (added
  separate item), and not worry about SO_KEEPALIVE unless we see more
  demand for it. It doesn't seem that nc110 or OpenBSD nc or so-called
  GNU Netcat support SO_KEEPALIVE either]

o [Ncat] In verbose mode, I'd like to see clock time (duration) and
  maybe in/out traffic stats when a client connection ends.  Maybe it
  could use a format similar to what Nmap provides. [David/Venkat]

o Seriously consider making --traceroute work even when we haven't
  found a probe which elicits a response from the target.  We'd just
  have to pick a probe in that case (probably echo request, as we
  found that to be the most effective in prev. empirical testing).
  This is similar to UNIX traceroute and Windows tracert.exe which
  just pick a probe (high UDP port on UNIX, ICMP echo request on Win).
  Even if the host is down or something, we usually get some useful
  hop information.

o [NSE] Allow spaces in script arguments without the user having to
  manually quote them (beyond normal shell escape quoting).  See:
  http://seclists.org/nmap-dev/2009/q3/0090.html
  [Patrick]

o [Ncat] Support SCTP now that Nmap does. 
  - See client support patch by Daniel Roethlisberger: 
    http://seclists.org/nmap-dev/2009/q2/0609.html
  - Server support?
  - Daniel has a patch, David looking to apply once an nsock thing is fixed.

o Look at etc/payloads.conf in unicornscan-0.4.7 and see if they have
  any which we don't have, but should, for our version detection.
  They have a decent collection there.  KX sent some other programs we
  should look at too. [David]

o Ncat should give it's ethernet cat ASCII logo after
  configure--similar to the way that Nmap, Ncrack, and Nping
  do. [David/SoC]

o [Zenmap] The Search dialogue is helpful for finding a certain scan
  you've performed recently, but we should probably also offer a similar
  function for searching for certain applications/hosts within a scan
  (e.g. find all the hosts running Apache).  This new functionality
  might be a find option or some other mechanism rather than being
  part of the Search dialogue proper.

o Ncat SSLv2 issues.  See
  http://seclists.org/nmap-dev/2009/q1/0319.html.  A big part of it is
  done, which was enhanced version detection probes to detect more SSL
  servers, The defect that remains is that Nsock can't connect to a
  small fraction of servers (including some of the ones detected by
  the new version probe). They are the servers that do only SSLv3 or
  TLSv1 and don't respond to a SSLv2-compatible ClientHello.  Even
  though most servers don't support SSLv2, they usually respond to the
  ClientHello and just don't offer any SSLv2 features. [David/Venkat
  working on this]

o Deadlock identification and correction:
 o Plan of action: implement freeing of script mutexes when scripts
 exit without freeing them (done and in /nmap now). And then if it
 continues to be a problem we'll consider this other stuff:
  o Add detection for deadlocks and print which threads are involved.
  o use above results to make a strategy for automatic deadlock resolution.
  o Original entry: Figure out what to do about NSE mutexes:
  http://seclists.org/nmap-dev/2008/q3/0276.html . In particular, they
  are not currently cleaned up if a thread dies or otherwise exits
  without unlocking them and can cause endless deadlocks which are
  annoying to users and can be difficult to debug :(.  Patrick has
  some ideas for this in his SoC09 proposal:
   "Adding a cleanup system for NSE that is called periodically
    similar to nsock_loop. There would be a registration system
    allowing C libraries to register a Lua function that will run
    periodically to check for irresolvable deadlock or simply dead
    resources. For example, the nmap library would register a mutex
    cleanup handler which would inspect all mutexes looking for a dead
    thread or circular dependencies. The nsock library could register
    a handler that checks for unused sockets. The nsock may save a
    strong reference to the thread that owns the socket and inspect it
    to determine if the thread is dead."
  David later says: "After some discussion we decided to start more
  modestly, first by ensuring that a scripts mutexes are released when
  it dies for whatever reason. I have a hunch that this is the cause
  of most deadlocks. It was certainly the cause of two whois.nse
  deadlocks I found. Then, the next step if deadlocks continue to be a
  problem, is to do automatic detection and just print out a list of
  what scripts are involved. It could be that several smb scripts are
  deadlocked, or as in the case I observed where whois.nse was locked
  with itself."

o Joao is auditing his Lua code to make sure all his variables are
  local where appropriate. [Joao - done, should be commited very soon]

o [NSE] We need to deal with libraries which improperly use global
  variables, as that is very common (Patrick made a list:
  http://batbytes.com/bad.txt).  Solutions could involve augmenting
  our runtime system (the "strict.lua" approach) to detect/prevent the
  problem, a script we run occasionally to identify issues that we
  then manually resolve, or, at the very minimum, documenting
  somewhere in scripting.xml the dangers inherent in global variables
  and warn people to generally declare them local instead.  We have a
  long history of bugs caused by non-local variables defined in NSE
  libraies and often causing deadlocks.

o The Nmap refguide (http://nmap.org/book/man-performance.html) says
  "The --max-parallelism option is sometimes set to one to prevent Nmap
  from sending more than one probe at a time to hosts. This can be
  useful in combination with --scan-delay (discussed later), although
  the latter usually serves the purpose well enough by itself."  But
  when you actually try it:
  # ./nmap --max-parallelism 1 --scan-delay 10 scanme.nmap.org
  You can't use --max-parallelism with --scan-delay.
  QUITTING!
  We need to either make that work or adjust the documentation. [David/SoC]
  o David changed this to a warning.  Note that with --scan-dealy,
  --max-parallelism is essentially 1 anyway.

o [NSE] Consider integrating HP Laserjet print PJL status-setting
  script.  See this thread for an example of such a script:
  http://seclists.org/nmap-dev/2009/q3/0083.html (note that it is
  updated during the thread).  Also, see this thread:
  http://seclists.org/nmap-dev/2009/q3/0092.html

o Ndiff man page should be expanded to include sample execution/output
  and more fully describe its functionality. [David]

o David is going to reexamine the old coverity-reported issues (the
  ones we previously marked as "ignore" because they weren't real bugs)
  just to be sure that is (and is still) the case.

o Make -sP work with -PN to disable both port and ping scanning.  We
  need to make sure the various options still work (-O, --script,
  --traceroute, etc.) with this, as many currently don't as they don't
  expect this behavior, which used to be unsupported and cause Nmap to
  quit with an error messaqge.  It may be OK to refuse -O since that
  will rarely give useful results.  OTOH, -O may work on some systems
  with unique closed port signatures where Nmap guesses a closed
  port. Users should then be able to do an NSE-only scan with "-sP -PN
  --script [scripts]" We should document this -sP -PN usage in
  refguide.  [David]

o Add -sn and -Pn options which are aliases for -sP and -PN.  Once
  they've been around long enough to be in most people's copy of Nmap,
  we plan to document those as the preferred version.  Those match -n,
  and the main problem with -sP is that we now use it more for
  "disable portscan" than ping only.  For example, you still might
  want to use NSE. [David]

o [NSE] Make sure all our HTTP scripts transparently support SSL
  servers too. [Joao has a solution and is testing the http scripts to
  make sure they don't break.]

o Resolve "memcpy overlap in getinterfaces(int*) (tcpip.cc:2987)".
  See this thread: http://seclists.org/nmap-dev/2009/q2/0713.html
  [David/Brandon]

o [Ncat] Print a message to stderr upon connection failure even if -v
  isn't specified so the user knows what went wrong. [David/SoC]

o [Ncat] Maybe --chat should imply -l.  And Maybe --broker should too?
  - OTOH, we might want to extend --chat for connect mode in the
    future.
  [We're going to hold off on chat now, David/SoC is doing --broker]

o Consider making it easier to tell whether scripts were specified by
  name on the command-line (rather than default or by class) so they
  have the option of providing extra verbosity in that case.  For
  example, see http://seclists.org/nmap-dev/2009/q2/0563.html.  We
  could either provide a special function for scripts to determine
  that, or we could magically adjust nmap.verbosity() when called by
  those scripts. [David]

o [NSE] Figure out a way to support people who want to do script scan,
  but not port scan or ping scan.  One option would be to allow
  --script to list scan (-sL), but perhaps a better option is to
  provide a way to disable port scanning in the same way as we offer
  -PN to disable ping scanning.  As an example of this need, David had
  to write special code to avoid ping/port scanning when doing a
  whois.nse survey for
  http://www.bamsoftware.com/wiki/Nmap/EffectivenessOfPingProbes.  The
  key for this task is to figure out how to do it from a user
  interface perspective and then implement and document it.  We've
  already been going in the direction of allowing script scanning in
  more types of scans--a while back we started allowing it with -sP
  ping scans due to high demand. [David/SoC]
  [ We decided how we're going to do it (-sP -PN to start out with;
  leading to eventual -sn -Pn) and added new TODO entries for actually
  doing the code/docs. ]

o Ndiff should be able to show NSE script result changes. [David]

o Get set up for Coverity scan of latest version to see if it catches
  any important issues before stable release. [Fyodor,David]
  [Found 7 new results, 3 are real bugs, and 2 have been fixed so far]
 
o [nsock] Fix Makefile to handle dependencies correctly (if that turns
  out to be the problem).  See
  http://seclists.org/nmap-dev/2009/q1/0629.html.  o Or it may be
  related to SVN timestampling.  See
  http://seclists.org/nmap-dev/2009/q1/0632.html. Diagnosed by David:
  http://seclists.org/nmap-dev/2009/q2/0728.html

o For at least our UDP ping probes, Nmap should probably notice if it
  is a very well known service port such as 53, 161, or 137 and send
  an appropriate probe packet (server status for DNS, public community
  string query for SNMP, etc) rather than empty data in that case.
  This is similar to the way our IP protocol probes automatically
  include common headers such as TCP and UDP if that common protocol
  is given.  Good probes for these services are already available in
  nmap-service-probes, though we might want to make a custom file for
  this.  We should probably do this for port scanning as well. [David]

o [NSE] Make NSE work better for SSL tunneled services in general by
  supporting them easily in the libraries.  For example, I don't think
  irc-info.nse currently works against all the servers which tunnel
  over SSL.  Maybe augment comm library, etc. [Joao - done, except for
  http, which is already a separate TODO item]

o Update scripts which use table args to use pseudo-table format
  "name.arg" rather than requiring the user to create a Lua table
  themselves. On the lua side, it's not really being stored in a
  table, but just an arg named "name.arg". [Joao]
  - Look at all our existing scripts which use tables
    (dns-zone-transfer, whois, the proxy scripts, etc.) and change as
    appropriate.  Remember to change the usage throughout the script
    and also change the nsedoc script arguments and example usage.
    For the existing scripts, try to retain the table version check
    for now to avoid breaing backward compatability if possible.  Just
    add the newer style check as well.
  - Is taking arguments in a table specific to a script a good idea?
    The example in the socks-open-proxy nsedoc of "--script-args
    openproxy={host=<host>}" is a bit of a mess and I'm not sure the
    best way to document that in the script argument list.  Note that
    this is the standard way we've handled it for some other scripts,
    so it's not an open-proxy-script-specific problem.

o [NSE] Track active sockets in the nsock library binding and don't
  rely on garbage collection for reallocation. Can probably wait until
  post-stable release for integration. [Patrick]
  - Patrick has a patch and is waiting on dev branch to check it in.

o [NSE] Resolve ssh2.lua buffering problems
  (http://seclists.org/nmap-dev/2009/q2/0673.html) [Joao]

o Decide what to do about ncat source code headers -- maybe just use
  the Nmap ones. [David added the Nmap headers]

o Once we go into deep stability freeze mode, create an nmap-exp
  development branches for changes we plan to integrate after the
  stable release. [Fyodor]

o Update CHANGELOG for latest changes [Fyodor]

o Release 4.85BETA10

o [NSE] Open proxy detection scripts
  o We have http-open-proxy.nse, but we should probably either extrand
  that to handle other types of proxies (such as SOCKS and HTTP
  CONNECT) or create more scripts to handle those other proxy
  types. [Joao, David]
  o Joao has written scripts, just need to finish up, evaluate, integrate.

o Determine whether zenmap.spec.in can currently require
  "python-sqlite" rather than "python-sqlite2", or if it at least can
  be easily made to do so. The former seems more compatible since
  RHEL/CentOS 5.3 has a "python-sqlite" package, but not
  "python-sqlite2".  Meanwhile, Fedora 10 provides the "python-sqlite"
  capability as long as you have the Python 2.5 package installed
  (python-2.5.2-1.fc10).  Fedora 10 does also make a
  python-sqlite2 package available.

o [Ncat] Solve EOF issues which crop up when piping to an external
  command. See http://seclists.org/nmap-dev/2009/q2/0528.html. It
  sounds like we will go with Daniel's patch [Daniel, David]

o Look into building RPMs with SSL support. Statically linking to
  OpenSSL on Linux for the RPMs didn't work for me last time I
  tried. [Fyodor]
  o Static linking of Nmap to OpenSSL does not seem to work on Fedora
    10 or CentOS 5.3.  The problem appears to relate to the OpenSSL
    krb5 support.
  o Could build my own OpenSSL libraries on the build system
    (w/o Kerberos support) and link to those.
  o At some point, we might want to consider including OpenSSL with
    Nmap tarball.  The problem is that it is rather big.  Would
    increase Nmap .tar.bz2 size from about 9 megs to about 12.  OTOH,
    OpenSSL is only going to get more and more important.  Maybe we
    can include a stripped down version?
  o If we don't integrate OpenSSL (or until we do), we might consider
    a more prominent configure warning for when SSL is not detected.
    We could suggest that users run "yum install libopenssl-devel" or
    "apt-get install libssl-dev" commands or whatever is appropriate
    and then reconfigure.  Or we could point them to a page or
    nmap-dev posting URL with instructions.

o Figure out why I [Fyodor] get a bunch of "Operation not permitted" errors
when I launch a scan on SYN such as: 
  - I'm going to ignore this for now unless it causes me trouble
  again, as this is an old machine that will be replaced soon anyway.
  And we haven't been hearing of the problems from others lately.
   /home/fyodor/nmap-exp/fyodor-perf/nmap -nogcc -T4 -n -v -p- --portpingfreq 250 -oA /home/fyodor/nmap-misc/logs/WorldScan/portpingfreq/logs/portpingfreq-250-1%T-%D 67.15.236.34 67.15.236.36 81.174.236.66 81.174.236.119 170.140.20.160 170.140.20.174 202.138.180.9 202.138.180.17 202.138.180.132 209.20.64.112
   The errors look like:
sendto in send_ip_packet: sendto(7, packet, 44, 0, 170.140.20.174, 16) => Operation not permitted
Offending packet: TCP 64.13.134.4:59820 > 170.140.20.174:59120 S ttl=39 id=19927 iplen=44  seq=2425535549 win=4096 <mss 1460>
sendto in send_ip_packet: sendto(7, packet, 44, 0, 67.15.236.36, 16) => Operation not permitted
Offending packet: TCP 64.13.134.4:59820 > 67.15.236.36:15030 S ttl=57 id=50640 iplen=44  seq=2425535549 win=2048 <mss 1460>
Discovered open port 49394/tcp on 170.140.20.174
sendto in send_ip_packet: sendto(7, packet, 44, 0, 170.140.20.174, 16) => Operation not permitted
Offending packet: TCP 64.13.134.4:59819 > 170.140.20.174:8256 S ttl=48 id=38510 iplen=44  seq=2425601084 win=1024 <mss 1460>
   May be related to connection tracking and high scan rates. See
   http://seclists.org/nmap-dev/2008/q4/0652.html
   http://www.shorewall.net/FAQ.htm#faq26
   Others have reported similar issues even without connection tracking.  See
   http://seclists.org/nmap-dev/2006/q3/0277.html
   http://seclists.org/nmap-dev/2007/q2/0292.html


o -PO1 and "-sO -p1" seem to send ICMP ping packets with an ICMP ID
  field of 0, which we found that a small percentage of hosts drop
  (61.13% responded with 0, 62% with a random value).  So we might as
  well randomize them in these cases. [Josh Marlow]

o Some of the -PS443 scans (and maybe other ones) we've been running
  have been missing the Nmap line telling how many packets were
  sent/received, even though we had verbose mode. [David/Josh]

o Deal with Ncat newline problem.  See this thread:
  http://seclists.org/nmap-dev/2009/q2/0325.html [David,Jah]

o Integrate SCTP scanning support.  See Daniel Roethlisberger's branch
  in nmap-exp/daniel/nmap-sctp.  As of 4/30/09, he is nearing
  completion.  See http://seclists.org/nmap-dev/2009/q2/0270.html.

o [NSE] Release mutexes upon script death to prevent certain deadlocks
  [Patrick, David]

o Consider whether to let Zenmap Topology graph export the images to
  svg/png/etc.  Also think about printing.  Note that João Medeiros
  has written a Umit patch to do this: [Joao, David]
  http://trac.umitproject.org/ticket/316.
  - Now he has Nmap patch:
    http://seclists.org/nmap-dev/2009/q2/0409.html
  - Consider integrating.
  - Integrated!

o Ensure that when I build a distribution package on UNIX (e.g. make
  distro), it builds what is in the Nmap directory I am calling it
  from rather than a particular SVN version.  I'm going to start
  building packages from a special "clean" directory which is
  different than the one I do development work in.  Also, I want to be
  sure that any changes in that dir are included in the release, even
  if they aren't check in yet. [Fyodor]

o Nmap UNIX distro build script should regenerate script.db. [Fyodor]
 o Now it is in make prerelease

o Nmap build system should be split into [Fyodor]
 o prerelease -> generates version files, man pages, script.db
   etc. That has to be done on one system, and then results checked in
   before doing a make release.  It does this stuff based on the
   directory it is run in rather than some set dirname or a pure SVN
   version
 o release-tarballs -> does any system-dependent building and creates
   the source tarballs.  It does this stuff based on the directory it
   is run in rather than some set dirname or a pure SVN version
 o release-rpms -> Same as above, but also uses the created tarballs
   to build the Linux RPM binaries for the current platform based on the
   tarballs.

o Build x86 and x86-64 VM instances for RPM building. [Fyodor]
 * I think I'll use CentOS 5.3

o [NSE] Script scanning does not seem to work on Fyodor's Linux
  machines after being installed from latest SVN (or 4.85BETA9) and run
  as a non-root user (it works fine as root).  The command "nmap -sC
  localhost" leads to NSE failure messages which differ based on the
  exact version run. [Was a relatively simple permissions problem in
  our Makefile.in -- I fixed it]

o [NSE] Release socket locks on connection failure or
  timeout. [Patrick]

o Update Nmap entry on Linux Online -
  http://www.linux.org/apps/AppId_1979.html
  - Screw it, the site does not seem to be maintained at all.  They
  aren't taking updates as of 6/2/09, and even Firefox shows latest
  update as 0.9.1.

o [Ncat] In verbose mode, print when an SSL connection is established
  successfully and give the leaf certificate hash to make it easier to
  verify when connecting to a machine where you can't or don't want to
  use --ssl-verify (e.g. connecting to an ncat ssl server where it
  created its own key).  While we're at it, we might want to print
  some other information from the leaf node, such as organizationName
  and maybe localityName, countryName or something.  We don't want to
  be too verbose, but 1 line would be great and 2-3 might be
  acceptable. [David]

o Fix NSEdoc to better escape single-quotes in fields.  If we can't do
  that for some reason, we need to document it better.  For example,
  when we initially tried generating nsedoc for
  http-webdav-unicode-bypass.nse, NSEdoc was listing it as a module
  named "s auxiliary module", apparently because this line exited in
  the description field:
  This module is based on Metasplit's auxiliary module, modules/auxiliary/scanner/http/wmap_dir_webdav_unicode_bypass.rb. 
  (For full example, see scripts/http-webdav-unicode-bypass.nse
  r13345) [David/SoC]

o --script-args should allow a wider range of characters, and should
  give a more useful error message if it receives chars it really
  can't handle for some reason.  For an example, try
  "--script-args=smbuser=admin,smbpass=pass^word".  For more details,
  see Ron's report at
  http://seclists.org/nmap-dev/2009/q2/0378.html.

o [Ncat] Have --ssl-cert and --ssl-key send a certificate in connect
  mode so that client certificate auth can be done. [David/Venkat]

o Once we're done with host discovery empirical research, add it to
  host-discovery.xml.  Would be great to show the best combinations to
  use for a given number of probes, the efficiency of the common probes
  by themselves, etc.

o Consider making the ping scan default be more comprehensive.  Note
  that I got 23% more Internet boxes found out of a 50K sample (see host
  enumeration chapter of my book for details).  Maybe I should
  experiment a bit more to ensure they are real boxes and not network
  artifacts and figure out exactly which tests are helping the most.
  If I do this change, I'll have to update the host enumeration
  chapter.  For UDP probing purposes, we should test whether including
  extra data in the packet (e.g. --data-length) helps in general, and
  for services such as 53 and 137, we should probably send proper
  protocol headers (e.g. a DNS server status message) so that we
  receive responses from listening services.

o We should probably check for a system Lua in a "lua5.1" directory
  rather than just "lua", as Debian and also my Fedora 10 systems seem
  to have that.  See
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=527997. [Note,
  Fyodor asked the bug reporter Jan Nordholz on 5/14/09 if he could
  write a patch. Jan sent in a patch, it worked, Fyodor checked it in.]

o [NSE] Get rid of ceil so that floating point NSE runlevels work
  again (some scripts, including (smb-brute) rely on this.  They got
  broken with the NSE core lua rewrite. [David].

o NSE script logical operator stuff is now documented in
  scripting.xml--add to refguide.xml as well. [David/Patrick]

o [NSE] Correct nsock_connect to unlock the socket slot if the
  connection fails.  When a socket is closed, it is unlocked so the
  arbitrator can potentially open up a socket for another thread. But
  Patrick discovered that a socket is not automatically unlocked when
  a connection fails or times out, only when it is closed
  explicitly. So that could hold up socket allocation for other
  threads until garbage collection.  May be a cause of slowness or
  possibly deadlocks. [Patrick]

o [NSE] Solve segfault issue which occurs when Nsock events call back
  on a thread that has already ended (e.g. timeout, crash, early exit,
  whatever) and been garbage collected.  May want to just nsi_delete
  all nsock sockets immediately upon thread ending.  For an example of
  this type of segfault, see
  http://seclists.org/nmap-dev/2009/q2/0289.html. David says " I think
  in the interests of getting this in a stable release, we should use
  that strategy of closing all a thread's sockets. That ought to fix
  all the problems above.  Not to rule out a more thoughtful redesign
  in the future." [David,Patrick]

o We added the SEQ.CI value in Feb 2009 with 0 matchpoints.  At some
  point (once we have some real-life values) we need to evaluate whether
  we want to give it points.  A good time to do that would be when we
  next do fingerprint integration, so we will actually have examples
  of .CI in the nmap-os-db. [David]

o [NSE] Make it a warning rather than error if a script in script.db
   can't be found. [Patrick]

o Add version detection signature for Ncat chat once we finalize the
  announce format. [David]

o Change Nmap signature files to use the .sig extension rather than
  .gpg.txt, as that seems to be what gpg recommends.  In fact, gpg
  will automatically verify the right file if it exists after dropping
  the .sig (or .asc) extension.  I may need to configure .htaccess to
  serve .sig files properly.  Update nmap-install.xml
  accordingly. Suggested by tic at eternalrealm.net by email on
  7/13/08. [Fyodor]
  * Rename existing files, add symlink from the old .gpg.txt to .asc
    versions
  * Add appropriate .htaccess content type if needed for downloads
   - not needed since I decided on .asc extension rather than .sig
  * Update the generation scripts
  * Update the book documentation -
    http://nmap.org/book/install.html#inst-integrity

o Ask Coverity if they'll scan latest version of Nmap. [Fyodor asked
  David Maxwell on 5/14/09 ]

o Make 4.85BETA9 release [Fyodor]

o [Zenmap] Make a way to start a scan from the profile editor without
  creating a profile, then remove the command wizard.  This is partial
  implementation of
  http://www.bamsoftware.com/wiki/Nmap/ZenmapCommandLine. [David]

o [Ncat] Make proxy server mode work on Windows (this is the last
  remaining fork() dependency in Ncat).

o Do an OS detection integration run -- last was based on
  1/8/09. [David]

o [Ncat] Maybe we should create an SSL cert with no passphrase during
  Ncat compilation or install process so that if someone specifies
  Ncat -l and --ssl with no --ssl-cert and --ssl-key, we already have
  one for them, and it is a slightly better one (since the private key
  isn't known) than if we distributed a key.  Obviously it is still
  subject to MITM attacks since there is no domain validation going
  on.  But people who need that will have to buy a key from a
  certificate authority in any case.  We could create the key by using
  the "openssl" command line tool as shown in
  http://nmap.org/ncat/guide/ncat-advanced.html#ncat-ssl, or maybe
  better to have a way for ncat to do it using openssl calls. [David]

o [Zenmap] Should probably give some sort of widget indication that a
  scan is running.  Now that we can start multiple scans at once, the
  "scan" button goes back to being unpressed while the scan is
  running.  As some scans take minutes or more to show output, it is
  not always clear whether they are still properly running.  We should
  probably have some sort of widget, such as the throbber used in web
  browsers, to show that Nmap is still running.  It could be fore a
  specific scan (kind of like how you have a separate throbber for
  each tab on a web browser), or a global one which means at least one
  scan is running.  Or maybe a different sort of indication is in
  order (like a timer). [David]

o Further investigate Nmap Proxy patch by Zoltan Panczel and Ferenc
  Spala.  See http://nmap-dev.fw.hu/ and
  http://seclists.org/nmap-dev/2009/q1/0255.html . [Discussed it and
  then added new proxy feature item]

o Wherever practical, fix compiler warnings when compiling Nmap with
  VC++ 2008 Express SP1 (there aren't many). [David]

o [NSE] Consider adding boolean expressions to --script arguments.  For
  example, see Patrick's implementation at
  http://seclists.org/nmap-dev/2008/q3/0300.html .

o Generate a list of trusted SSL certificates to ship with Ncat (by
  extracting f rom Mozilla or similar), and install them with
  Ncat. Decide how these certificat es should be preferred to any
  system-provided certs, if any. [David]

o [NSE] Add desired SoC09 infrastructure ideas to this TODO to the
  extent they don't already exist.

o [Ncat] Consider supporting server certificate verification when used
  in client SSL mode.
  o For now we document in user's guide that it is not secure.
  o Maybe we can do an ssh-style approach where we just print the
    fingerprint and expect the ncat client user to ensure it is the
    right one?
  o If we're going to verify cert's etc., we need to also make sure we
    are actually using secure ciphers.  We may need to update nsock to
    support cipher selection, because we want fast ones for version
    detection, but usually want secure ones for NSE and/or ncat.
  o Do we want to check all this by default, or offer an option for
    it?  Doing it by default is more secure, though it can be annoying
    when a certificate has expired, is self-signed, you connect to
    domain.com when the certificate is for www.domain.com, etc.  If it
    is done by deault, we might just print an error message.  Whreas
    if we have a special option, it may be OK to exit and refuse the
    connection.
  o What certs should we allow?  Same as the browsers do?  Maybe get
    rid of Comodo?  Maybe we should fail to recognize any certs with MD5
    in the trust chain?
  o What about people who are running their own SSL service and just
    want to specify the cert file they use, because they generated it
    themself and not from a trusted CA.
  o Need to check expiration, domain, etc. if we're checking certs at
    all.
  o We can probably get away with not doing revocation checking, as
    long as we document that we don't.

o consider changing status field from "up" and "down" to "online" and
  "offline".  Actually, maybe we don't want this after all.
  online/offline look pretty similar, and they're longer too.  I'm
  taking this out of the TODO.

o [Ncat] When acting as an HTTP proxy, we should support GET mode as
  well as CONNECT so that it works as a non-SSL proxy in browsers such
  as firefox. [David]

o Finalize GSoC applicant research, communication, and selection
  [David, Fyodor]

o Go through all the SoC applicants and decide who we want to accept
  and start communicating with them. [David,Fyodor]
 o Decide which applicants we want, and who would be best for
    mentoring them.

o Document that U1.RID gives "G" as long as all the data bytes in the
  echoed response data are "C" as expected.  This G code is still
  given even when the response is truncated, including if there are 0
  bytes echoed. [David]

o [Ndiff] Rethink the output format. David says: In particular, I
  would like to always have the old state on the left and the new
  state on the right: "was filtered, is open," not "is open, was
  filtered." I also like the context diff output of MadHat's
  nmap-diff. [David]


o Canonicalize the "host up" messages for port scan and ping scan so
  that instead of things like "Host scanme.nmap.org (64.13.134.52)
  appears to be up ... good." we standardize in both cases on
  something like: "Host scanme.nmap.org (64.13.134.52) is up (.75s
  latency)".  Note the addition of the latency value, which is our
  srtt value for the host.  This will only show in ping scan and
  verbose port scan because the line doesn't appear without verbose
  mode. [David]

o Ping scans always seem to say "0 [hosts] undergoing Ping Scan" when
  you request stats, rather than the proper number.  For an example,
  try a command such as "nmap -iR 10000 -sP -n" and then press enter
  during the scan.  Here are some examples of the bad output: Stats:
  25:34:33 elapsed; 991232 hosts completed (72530 up), 0 undergoing
  Ping Scan Ping Scan Timing: About 53.69% done; ETC: 22:49 (0:00:09
  remaining) Stats: 0:01:10 elapsed; 0 hosts completed (0 up), 0
  undergoing Ping Scan Ping Scan Timing: About 24.03% done; ETC: 22:42
  (0:03:41 remaining) Stats: 0:03:28 elapsed; 4096 hosts completed
  (284 up), 0 undergoing Ping Scan Ping Scan Timing: About 3.06% done;
  ETC: 22:44 (0:03:07 remaining) [David]


o Remove obsolete tests from nmap-os-db itself. [David]

o Prepare for Summer of Code
 * Brainstorm for ideas
 * Create new ideas page
 * Apply to participate in program again
 * Advertise for applicants
 * Evaluate applicants

o NSEDoc script/module documentation pages should probably provide a
  link to the script/module source code (except for C modules).  The
  link format should probably be of the form
  http://nmap.org/data/scripts/[script].nse and
  /data/nselib/[module].lua.  NSEdoc can assume they already exist
  there, as we'll probably put them there using the same system we use
  to copy other stuff to the data dir.

o [Ncat] Let people set up authenticated proxies using
  --listen and --proxy-auth together (right now we don't support
  that). [David]

o When you specify multiple comma-separated arguments to --script,
  those arguments seem to get lost when the Nmap command is printed in
  Nmap's output files.  For example, I run the command:
  nmap -oN - --script=discovery,intrusive scanme.nmap.org
  The output includes:
  # Nmap 4.85BETA4 scan initiated Thu Mar 26 15:40:05 2009 as: ./nmap
    -oN - --script=discovery scanme.nmap.org
  Note the missing ",intrusive" in the script argument. [David]

o Merge patrick/nse-lua-merge for easier-to-maintain and simpler
  codebase once David and Patrick are happy with it. [David]

o SVN check out /nmap as an external in a directory named svn or src
  or nmapsvn or something under nmap.org web tree.  Then redirect the
  individual nmap.org/data/ files, where needed, to the nmapsvn
  instead.  and update nmap-dev Makefile not to copy them to the
  /data/ dir anymore. Then update the nsedoc system to generate proper
  links to the new script/nselib locations. [Fyodor]

o Improvements to presentation of version detection
  information. [Brandon]
  o Allow longer strings.  Right now it can be 128 chars for the
    fullversion info, I think.  But that isn't enough for this useful
    information-packed string: "Apache httpd 2.0.52 ((Red Hat)
    mod_perl/1.99_16 Perl/v5.8.5 DAV/2 mod_jk/1.2.19 PHP/4.3.9
    mod_python/3.1.3 Python/2.3.4 mod_ssl/2.0.52 OpenSSL/0.9.7a)".
    After discussion w/Brandon, we're going to allow 160 chars total.
  o Instead of omitting all information when version info string too
    long, we're going to truncate and allow 157 characters, plus
    ellipses (...)
  o Brandon says: "my final gripe is that the full version string is
    constructed as <product><space><version><space>(<extrainfo>).
    but, even if product or version are blank, the spaces are still
    there"

o I need an output-autoflush option of some sort.  This could be
  useful to ensure I get all the --packet_trace and debug data before
  Nmap crashes.  Actually, I'm not sure that is so critical.
  o Killing it for now, not sure that it even is needed.

o Fix the directory function(s) in nse_fs.cc to be usable by scripts and
  improve flexibility. [this entry added by Patrick]

o [Ncat] The sys_wrap.c/.h code contains a whole bunch of capitalized
  versions of system calls (Fork(), Socket(), Sscanf(), etc.) which
  are mostly the same as the standard version except that they cause
  ncat to quit if they are triggered.  They also may be used partially
  for portability.  The main issues are:
   1) Because the function quits in the case of errors, it doesn't
      always have the context to print a useful error message (and
      even when it does, it often doesn't -- for example Fopen could
      print the filename, but doesn't.)  Also, sometimes these
      functions are called when quitting really isn't the desired
      outcome of an error.
   2) Some could be replaced by code in nbase, for example, Malloc
      basically does the same thing as our safe_malloc already used
      throughout Nmap.
  So we should probably consider simplifying/removing this code to the
  extent possible.  But we need to remember to add error detection to
  the callers where necessary rather than blindly switching from
  (e.g.) Connect() to connect(). [Kris or David]

o With --version-trace (may be a problem with other uses of nsock
  tracing too), I often get dozens of "wait_for_events" reports in a
  row in a very short period, flooding the logs.  For example, with
  the command "nmap -sV --version-trace www.google.com", I get:
  NSOCK (22.3570s) Callback: WRITE SUCCESS for EID 283 [74.125.19.147:443]
  NSOCK (22.3570s) msevent_delete (IOD #4) (EID #283)
  NSOCK (22.3570s) wait_for_events
  NSOCK (22.3570s) wait_for_events
  NSOCK (22.3570s) wait_for_events
  NSOCK (22.3570s) wait_for_events
  NSOCK (22.3570s) wait_for_events
  NSOCK (22.3570s) wait_for_events
  NSOCK (22.3570s) wait_for_events
  NSOCK (22.3570s) wait_for_events
  NSOCK (22.3570s) wait_for_events
  [Goes on for pages]

o NSE memory issues (and gh_list assert failure) [David]
  o See this thread: http://seclists.org/nmap-dev/2009/q1/0532.html
  o We're taking this out for now since the new nse-lua-merge
  tenatively looks like it fixes this.

o [Ncat] Why does Ncat require enclosure in a while loop to answer
  repeated UDP queries, but not TCP?  For example, see the "Emulating
  Diagnostic Services" section of the Ncat user's guide.
  o Note: http://seclists.org/nmap-dev/2009/q1/0133.html

o Determine what we should do about the IE.DLI OS detection test [David]
  o All of the 1656 results for this test in nmap-os-db are DLI=S.
    o Is the test not working right (producing the proper results
      against targets), or is it just a generally useless test for
      which virtually all targets respond the same way?
  o Are there other "useless" tests in nmap-os-db?  It is worth
    checking, IMHO.
  o We're going to get rid of IE.DLI, IE.SI, U1.RUL, and maybe TOS and
    TOSI tests.

o When you do ncat -h, Ncat should probably show the Nmap version
  number rather than (currently) 0.2.  Also ncat in -v mode should
  show that same header. [David]

o Ncat verbose mode (-v) should probably only give important messages,
  such as perhaps a message once you connect successfully to a port,
  or a message if the connection attempt times out.  An Ncat version
  banner (with URL) like Nmap has might be warranted (in verbose
  mode).  Currently, Ncat floods you with (mostly) useless debugging
  information like this with a single -v (this output, on the other
  hand, might be useful for a debugging option): [David]
  # ncat -C -v scanme.nmap.org 80
  NSOCK (0.0000s) TCP connection requested to 64.13.134.52:80 (IOD #1) EID 8
  NSOCK (0.0200s) Callback: CONNECT SUCCESS for EID 8 [64.13.134.52:80]
  NSOCK (0.0200s) Read request from IOD #1 [64.13.134.52:80] (timeout: -1ms) EID 18
  NSOCK (0.0200s) Read request for 0 bytes from IOD #2 (peer unspecified) EID 26
  GET / HTTP/1.0
  NSOCK (4.4280s) Callback READ SUCCESS for EID 26 (peer unspecified) (15 bytes)
  NSOCK (4.4280s) Write request for 16 bytes to IOD #1 EID 35 [64.13.134.52:80]
  NSOCK (4.4280s) Callback: WRITE SUCCESS for EID 35 [64.13.134.52:80]
  NSOCK (4.4280s) Read request for 0 bytes from IOD #2 (peer unspecified) EID 42
  For comparison, here is what Eric Jackson's nc (The nc available in
  Fedora 10's package repository) shows in verbose mode for the same
  connection:
  # nc -v scanme.nmap.org 80
  Connection to scanme.nmap.org 80 port [tcp/http] succeeded!
  GET / HTTP/1.0 [David]

o Final polishing of our GSoC pages. [Fyodor]

o Advertise widely for Nmap GSoC applicants [Fyodor]

o [Ncat] We should (maybe) consider a way for people to choose
  usernames in --chat.
  o Removing this for now.  We can add it back if we decide we really
    want this.

o Deal with new Python 2.6 Zenmap build warnings:
  C:\Python26\lib\site-packages\py2exe\build_exe.py:16: DeprecationWarning: the sets module is deprecated
  import sets
  http://sourceforge.net/tracker/index.php?func=detail&aid=2314799&group_id=15583&atid=115583
  [Bug in py2exe, will probably be fixed with a new version of py2exe
  once it is released and we upgrade.  This isn't causing us any major
  problem anyway.]

o When I scan large groups of hosts with OS detection enabled, I get
  groups of warnings like:
  Insufficient responses for TCP sequencing (0), OS detection may be less accurate
  Insufficient responses for TCP sequencing (0), OS detection may be less accurate
  Insufficient responses for TCP sequencing (0), OS detection may be less accurate
  Insufficient responses for TCP sequencing (0), OS detection may be less accurate
  Insufficient responses for TCP sequencing (0), OS detection may be less accurate
  Note how it doesn't even tell the relevant IP address, and it isn't
  included in an individual host section.  We should probably either
  include it in the section for an individual host, like we do with
  "OSScan results may be unreliable because we could not find at least
  1 open and 1 closed port", or (not quite as
  good) include the relevant IP address in the error message.  And we
  may or may not want to require verbose mode.

o Ncat chat should bomine the "already connected" user list into one
  line, like:
  <announce> already connected: 69.232.238.42 is connected as <user5>, 206.81.65.43 as <user4>, 69.232.238.42 as <user6>

o [Ndiff] Maybe Ndiff should display changes to version detection and
  OS detection information? [David]
 o Version detection done, now just needs OS detection.

o When I start ncat chat with this tcsh command:
  ncat -l --chat scanme.nmap.org < /dev/null >& /dev/null &
  The first client to connect to the chat becomes user0 and doesn't
  work quite right.  Messages user0 type get transmitted to other
  clients, but user0 does not see their messages.  Nore does user0 get
  the normal connection announcement upon connecting.  If I quit
  user0, the next client to connect becomes user0 again and has the
  same problem.  If I start ncat on the server with "ncat -l --chat
  scanme.nmap.org" (no redirection), other clients can connect with no problems.

o Ncat --chat should probably announce to everyone (including the new
  person) when someone connects.  This tells the new person their
  username, and lets everyone else know about the new connection. [David]
  o We should also tell the new person (and possibly everyone on the
   channel) the list of existing participants.

o SoC ideas page [Fyodor]

o Nmap 4.85BETA4 release [Fyodor]

o [Ncat] Wouldn't it be nice if we could support --exec (and maybe
  some sort of partial-emulated --sh-exec) on Windows? [David]
  o Almost working!  We found some problems with "ncat.exe -v -l
  --sh-exec "ncat -v scanme.nmap.org"

o [Ncat] Can we use it as an IPv4 <-> IPv6 gateway?  If so (or if we
  can add it), it should be added to the ncat guide feature list.
  o Yes, David tried it with --sh-exec and it worked.

o [Ncat] We should probably make it work without OpenSSL.  When I try
  ./configure --without-openssl on latest svn Nmap, Ncat build fails
  with:
       gcc -MM -I../libpcap -DHAVE_CONFIG_H -D_FORTIFY_SOURCE=2 -I. -I.. -I../nsock/include/ -I../nbase ncat_main.c ncat_connect.c ncat_core.c ncat_listen.c ncat_proxy.c ncat_broker.c ncat_hostmatch.c ncat_ssl.c util.c sys_wrap.c > makefile.dep
       make[2]: Leaving directory `/mondo/fyodor/nmap/ncat'
       make[2]: Entering directory `/mondo/fyodor/nmap/ncat'
       gcc -I../libpcap -DHAVE_CONFIG_H -D_FORTIFY_SOURCE=2 -I. -I.. -I../nsock/include/ -I../nbase  -c ncat_main.c -o ncat_main.o
       ncat_main.c: In function ‘main’:
       ncat_main.c:536: error: ‘struct options’ has no member named ‘ssl’
       ncat_main.c: In function ‘ncat_listen_mode’:
       ncat_main.c:646: error: ‘struct options’ has no member named ‘ssl’
       ncat_main.c:646: error: ‘struct options’ has no member named ‘sslcert’
       ncat_main.c:646: error: ‘struct options’ has no member named ‘sslkey’
       make[2]: *** [ncat_main.o] Error 1
       make[2]: Leaving directory `/mondo/fyodor/nmap/ncat'
       make[1]: *** [ncat_build] Error 2
       make[1]: Leaving directory `/mondo/fyodor/nmap'
       make: *** [static] Error 2

o [Ncat] Defensive coding review of Ncat --chat (talk)

o [Ncat] As SSL server it should not crash when someone connects in
  w/o SSL and does ^C.  When David tried it during our chat, the ncat
  servr "ncat --broker --ssl-key test-key.pem --ssl-cert test-cert.pem
  --ssl --chat -l"  crashed with: SSL_accept():
  error:00000000:lib(0):func(0):reason(0).  Also, when a Windows SSL
  clients joined and then left, the server died with "Broken pipe

o [Ncat] --chat should probably only allow reasonable chars, to avoid
  cntrl-chars, etc.

o Nmap should treat ports named "unknown" in nmap-services the same
  way (from a naming perspective) as it treats ports which are not
  listed at all.  See http://seclists.org/nmap-dev/2009/q1/0589.html.

o Ncat user guide "Emulating Diagnostic Services" page has a very long
  UDP chargen server line which causes wrapping problems in web browsers
  (e.g. it widens the page substantially).  It should probably be
  split into multiple lines. [David]

o Ncat user guide proxying section says "The only exception is when
  listing a proxy host by IPv6 address; then the port is required."
  Why would we require a port number for IPv6 rather than just use the
  same defaults as we do for IPv4?
  [David explained that this is because to do otherwise would be
  ambiguous because IPv6 uses : for separaters, so we wouldn't know
  how to handle things like FF::10:80]

o [Ncat] Perhaps we should make --ssl work in --chat.  If nothing
  else, it might be useful if you want to reduce the number of people
  connecting with telnet, etc. rather than ncat.

o [Ncat] --talk should probably be changed (in the code and
  documentation) to --chat, as Ncat chat has a
  much nicer ring to it, IMHO.  --talk should remain as an alias to
  --chat, but we don't need to document it. [David]

o Ncat Windows issue where you make a connection and then take several
  seconds to type in a line to the server, Ncat wrongly times out when
  trying to write your line to the remote server. [David]

o Ncat write timeout problems cause client to quit due to write
  timeout sometimes. [David]
  Examples:
   o yes | ncat localhost
   o when we paste a few lines into the terminal window in an Ncat chat

o Defensive coding review of ncat_proxy.* [David]

o Process the latest version detection submissions.  We now have more
  than 1,700 of them queued up. [Doug]

o Write Ncat users' guide, demonstrating all the neat stuff you can do
  with it.  This should probably be in DocBook XML so it can be an NNS
  chapter.  You might want to query nmap-dev for list of neat things
  people do with ncat (or look around for what people do with nc).
  Testing it out for examples might expose areas for improvement as
  well. [David]

o Look at Dario Ciccarone's email from 5/1/07 about IPID sequence
  issues, and consider adding IPID sequence test for closed-port-tcp as
  they apparently can be different. [David]
  o Also fix bug which causes SEQ to not be printed if the TCP open
    port tests fail to produce results, even though the II and
    (upcoming) CI tests may have useful results. [David]

o NSE should offer some way to sleep/yield for a given amount of
  time.  This would allow other scripts to run while a script has
  nothing to do.  Possible uses:
  o Many services have rate limits (or you might just want to use them
    for politeness).  For example, a web site spidering application
    might want to limit HTTP requests to some number per second to avoid
    pissing off the target webmaster more than is necessary (or prevent
    getting auto-blocked).  Similarly, whois servers often will block
    IPs which query them too often in a short period.  Or maybe you
    don't want to exceed the threshold limits of an IDS.
  o Example current scripts which might benefit: sql-injection, whois
    (possibly), pop3-brute, etc.
  o If we don't currently have a way for a cpu-bound NSE script to
    yield, then perhaps this could help us implement such a mechanism.
    But maybe coroutine.yield already does the trick.
  o The mechanism needs to be documented, and ideally should be
    implemented in at least one of the scripts shipped with Nmap.

o Consider adding a way for requesting timing status updates at a
  given interval (such as every 5 seconds) to XML and/or normal
  output.  This would be useful for people who run Nmap from scripts
  or other higher level applications. [David]

o Ncat --allow/--deny bug: "--allow and --deny only support host
  specification by IP address, and give no warning when you use
  another form such as a host name." Should probably use same syntax
  as --exclude. We also want to at least do verification at the
  beginning to make sure all the entries are legitimately formed.  We
  probably want to do things like DNS resolution at the beginning
  too. Otherwise we might have a DNS failure when we actually get a
  connection and perhaps have to reject the connection wrongly, or
  risk a false negative.  [David]

o Fix this overflow:
  Stats: 93:57:40 elapsed; 254868 hosts completed (2048 up), 2048 undergoing UDP Scan
  UDP Scan Timing: About 11.34% done; ETC: 03:21 (-688:-41:-48 remaining)
  [Done by David and Henri Doreau]

o Ncat -- perhaps connection brokering should support UDP as well as
  (its existing support for) TCP?  Actually this does raise issues
  such as deciding what list of UDP systems to forward a packet too.
  Its obviously not like TCP where you have a list of open
  connections.  Ncat could build such a list, but, for example, would
  never know when to remove the host.  For now, David is just going to
  adjust the error message to encourage people to email nmap-dev
  describing their usage scenario if they want this feature.

o Ncat documentation should note that no SSL certificate verification
  is done (maybe we should offer an option to do so, if OpenSSL makes
  that easy).
  o Done in the new Ncat user's guide

o Fix dns-zone-transfer infinite recursion bug described at
  http://seclists.org/nmap-dev/2009/q1/0317.html.  It sounds like the
  best approach is to use our dns.lua library rather than having
  dns-zone-transfer do its own DNS packet parsing.

o Fix XML escaping issue so that improper chars from NSE scripts or
  elsewhere can't cause corrupt XML files.  See
  http://seclists.org/nmap-dev/2009/q1/0316.html for an example. [David]

o Look into whether we should increase the frequency of port scan
  pings.  See http://seclists.org/nmap-dev/2008/q1/0096.html . Note
  that Fyodor already increased them a bit in 2008.  Might not need
  more.  [David did extensive testing of this one already]

o Find way to document NSE library script arguments and perhaps have
  them bubble up to scripts themselves.  For example, I had to read
  the SNMP library source code to determine the script argument to
  specify the SNMP community name for snmp-sysdescr
  (http://nmap.org/nsedoc/scripts/snmp-sysdescr.html).  Maybe we could
  just standardize on something like we do with SMB library and the
  scripts which call it (http://nmap.org/nsedoc/modules/smb.html,
  http://nmap.org/nsedoc/scripts/smb-check-vulns.html). [David]

o If it wouldn't bloat things too much, it would be nice to include
  ndiff in the Nmap win32 zip distribution files.

o Reported NSE crash:
  "Assertion failed - file ..\nse_main.cc line 314
   lua_gettop(L_script_scan) == 0"
  o He says: "After looking at this closer, it appears the assertion
    occurs if I include the IP where the scan is run from. For us, I'm
    running this on IP 57, which is a VMware Windows Server image.  If
    I eliminate that IP from the range it successfully completed the
    scan for all other devices."
  o Seems to be fixed.  He can no longer reproduce the problem with
    4.85BETA3.

o Deal with GTK DLL problem with Nmap 4.85BETA1: [Fyodor]
  o David's installer seems to work--he's using a different GTK
    distribution.  I'll try that.  Works!  Done!
  o Details on problem: http://seclists.org/nmap-dev/2009/q1/0207.html
  o Quick workaround done for 4.85BETA2, but better solution needed.

o "SCRIPT ENGINE (250.600s): ./scripts/rpcinfo.nse against
  a.b.c.d:<port> ended with error: ./nselib/datafiles.lua:114: attempt
  to index global 'arg' (a nil value)"
  -- http://seclists.org/nmap-dev/2009/q1/0227.html [Patrick]

o Consider making the TODO list public
  o Done: http://seclists.org/nmap-dev/2009/q1/0175.html
  o Probably remove all of the "done" items since that is easier than
    reviewing them.
  o Might as well add to insecure.org/nmap/data/
  o Maybe a bug tracker is a better approach.

o [NPING] Fix compilation on Solaris.  See
  http://seclists.org/nmap-dev/2010/q1/870.

