diff -PBbruw linux/Documentation/Configure.help magnet/Documentation/Configure.help --- linux/Documentation/Configure.help Thu May 24 16:03:06 2001 +++ magnet/Documentation/Configure.help Wed Sep 26 13:08:03 2001 @@ -1846,6 +1846,33 @@ You can say Y here if you want to get additional messages useful in debugging the netfilter code. +MAGNeT Network Instrumentation (EXPERIMENTAL) +CONFIG_MAGNET + MAGNeT: the Monitor for Application Generated Network Traffic. + + This is kernel-side code which instruments the Linux network + subsystem. It records incoming and outgoing packets at various + levels in the network stack(s), and records some pertinent details + of the packets (timestamps, etc). It is of use only to protocol + designers, researchers, and implementors, and should be disabled + by all normal people. Say N here. + + See Documentation/magnet.txt for information on how to use MAGNeT. + +MAGNeT Extended Instrumentation +CONFIG_MAGNET_EXTENDED + This enables MAGNeT to store extended information (ie, the entire + TCP state snapshot) with each data point. This significantly adds + to the overhead of the instrumentation, and thus should not normally + be used. Its here if you really need it. + +MAGNeT: Socket Calls Only +CONFIG_MAGNET_SOCK_ONLY + For the absolute minimum of instrumentation overhead. This tells + MAGNeT to only record application socket send() and recv() calls. + Thus, individual protocols (TCP, IP, etc) will NOT be monitored if + you say Y here. + IP: connection tracking (required for masq/NAT) CONFIG_IP_NF_CONNTRACK Connection tracking keeps a record of what packets have passed @@ -17786,6 +17813,7 @@ # # This is used by Emacs' spell checker ispell.el: # +# LocalWords: MAGNeT recv # LocalWords: CONFIG coprocessor DX Pentium SX lilo loadlin HOWTO ftp metalab # LocalWords: unc edu docs emu README kB BLK DEV FD Thinkpad fd MFM RLL IDE gz # LocalWords: cdrom diskless netboot nfs xzvf ATAPI MB ide pavia rubini pl pd diff -PBbruw linux/Documentation/magnet-install.txt magnet/Documentation/magnet-install.txt --- linux/Documentation/magnet-install.txt Wed Dec 31 17:00:00 1969 +++ magnet/Documentation/magnet-install.txt Wed Sep 26 13:08:03 2001 @@ -0,0 +1,99 @@ + +Step-by-Step instructions (sorta) on how to install MAGNeT. + +First, you need a MAGNeT-ized kernel. To do this, you need to reconfigure +a Linux kernel to support MAGNeT. First, get an official kernel release (i.e., +2.4.3) from www.kernel.org, or your favorite Linux kernel repository. Then, +download the appropriate MAGNeT patch for your kernel version. Uncompress and +untar the kernel, then do a "cd linux" and "patch -p1 < ". +Now, you are ready to configure the kernel. Some tips from Brian Bowers: + +1. Before attempting to configure kernel, find out what hardware is installed. +There is a "hardware inventory" script available on the web that is useful for +this - hinv. You can get it from http://reality.sgi.com/raju/software.html + +2. Seems simple, but remember to get and keep a copy of the original kernel +configuration (normally /usr/src/linux/.config). This can be used as a basis +from which to build the new kernel. + +3. CHECK "YES" when asked about being prompted for "Experimental portions". +Then the options for including MAGNeT appear under "Network Options". +This works in xconfig and menuconfig and just plain config. + +4. When done with the config, make each of the following: dep, clean, +bzImage, and modules. All of these can be done without root privileges. + +5. Have to have root privileges from here on, so get root. + +6. Copy file linux/arch/i386/boot/bzImage to /boot/mag +I don't want to destroy the original kernel until I'm sure that this kernel +boots correctly. + +7. Make modules_install. This will copy the module files to the +right place. (Experienced kernel builders will know this ...). + +8. Edit /etc/lilo.conf to add a section for the new, MAGNeT kernel. I copied +freely from the default section of my machine's lilo.conf to get the following (your actual mileage may vary :) + image=/boot/mag + label=MAGNeT + root=/dev/hda2 + vga=788 + append=" quiet" + read-only + +9. Run lilo + +10. Reboot the machine, to test the MAGNeT kernel. If the machine boots and +all else works, do a "cat /proc/net/magnet" to see if MAGNeT is installed. +(the file will exist if MAGNeT is properly installed, otherwise it won't!) + +11. OPTIONAL: go to the kernel source directory. make bzlilo to install the +MAGNeT kernel as the default kernel. This assumes that the MAGNeT kernel +works correctly. + +-- + +Once you have a MAGNeTized kernel up and running properly, you are ready to +finish the installation of MAGNeT. First, the MAGNeT user application files +need to be created -- this is done by changing to the Linux kernel root dir +(i.e., "cd linux") and typing the following commands: + +make net/magnet/magnet-read +make net/magnet/magnet-parse +make net/magnet/mkmagnet + +These files, along with net/magnet/magnet.cron and net/magnet/magnet.copy +should be copied to a global executable directory (such as /bin). Now MAGNeT +is installed and ready to run! + +--- + +To use the scripts that came with MAGNeT, modify the first several lines of +magnet.cron and magnet.copy to reflect your actual environment. You will need +to have SSH privileges to a directory on a central file share in order to +copy the MAGNeT data off of your machine. + +(RADIANT team note: The central file share in question is radiant-server. +Since we are running things in root's crontab, we need to copy the identity.pub +from our machine's root account to the end of root's .ssh/authorized_keys file +on radiant-server. This lets root SSH (and, therefore, SCP) without a +password. Then, we need to make a directory for our machine in +/array0/data/magnet. Set the owner to be your account name, group "radiant") + +Once these are set up, you are ready to enable magnet.cron to periodically +run -- simply do a "crontab -e" and add the following lines (appropriately +modified for your environment): + +# Set up MAGNeT to run every day at 12:34 AM +34 0 * * * /bin/magnet.cron + +If you want MAGNeT to start at boot-time as well, place a symbolic link in +the appropriate rc directory to magnet.cron. For instance, + +cd /etc/rc3.d +ln -s /bin/magnet.cron S89magnet + +You should probably also run magnet.cron manually right now, to get things +started (and make sure it runs successfully). On the first run, it will +complain about /dev/magnet not existing; if run as root, it will make the +device file and proceed happily.... diff -PBbruw linux/Documentation/magnet.txt magnet/Documentation/magnet.txt --- linux/Documentation/magnet.txt Wed Dec 31 17:00:00 1969 +++ magnet/Documentation/magnet.txt Wed Sep 26 13:08:03 2001 @@ -0,0 +1,90 @@ +MAGNet - Monitor for Application Generated Network Traffic +---------------------------------------------------------- + +MAGNeT is a research tool created by Los Alamos National Laboratory's RADIANT +(Research and Development in Advanced Networking Technology) team. The purpose +of MAGNeT is to obtain an understanding of the actual network traffic patterns +as that traffic traverses the networking stack in the kernel. While +traditional traffic monitoring solutions (ie, network sniffers) are capable of +reporting traffic on the wire, this may or may not reflect the actual traffic +patterns that applications generate -- for example, several small packets might +be queued up by TCP before being sent across the network. MAGNeT is the first +tool designed to show application-level rather then wire-level traffic. + +This is accomplished by a small bit of code in the Linux kernel. At strategic +points in the processing of network traffic (for example, when the TCP code +hands a packet of data to the IP layer, or when the kernel socket code hands +data from the TCP layer to the running application), a quick routine is run +to add a timestamp to a permanent, non-swappable kernel circular buffer. Since +this buffer space is guaranteed to be in memory, the marking has minimal impact +on networking performance. To read the recorded data, an user-level +application simply mmaps() the kernel circular buffer and reads out of it as it +is updated. + +This architecture allows MAGNeT to be used for any type of BSD Socket-based +communication. Currently, general socket calls (read(), write(), etc) and the +TCP and IP layers are instrumented. Additionally, the TCP and IP run-time +state snapshots can be optionally recorded at each timestamp. (This, of course, +results in much more data being moved around, and thus a bigger performance +impact) In this mode, MAGNeT can be used by protocol designers and +implementors to debug the workings of their protocol under known conditions. + +MAGNeT currently consists of the following: + +include/linux/magnet.h - + Header file to describe MAGNeT data types, for both kernel + and user-level programs. Also defines kernel function + prototypes. + +net/magnet/magnet.c - + The code to maintain the kernel circular buffer and + perform MAGNeT initialization and tear-down. + +net/magnet/magnet-read.c - + Sample user-level program to read MAGNeT data from the + kernel buffer. This sample program works by mmapping a + pre-existing file and transferring filesize bytes of data + from the kernel to the file. Rather simple, but usable. + +net/magnet/magnet-parse.c - + Program to read binary data collected from the kernel + buffer (ie, by magnet-read.c) and display the data in a + human-readable form. The timestamps reported are the CPU + cycle counter (which ticks once per CPU clock/instruction). + +net/magnet/mkmagnet.c - + Simple program to create the files which magnet-read.c + reads kernel data into. + +net/magnet/magnet.cron & net/magnet/magnet.copy - + Simple shell scripts to automate running the above user + space program to collect and archive data. magnet.cron + also takes care of creating the magnet device file (see + below) + +The following kernel compilation options are available: + +CONFIG_MAGNET + Compile in the base MAGNeT code + +CONFIG_MAGNET_EXTENDED + Compile extended MAGNeT data (protocol state snapshots, etc) + +CONFIG_MAGNET_SOCK_ONLY + Normally, MAGNeT records data for all protocols for which MAGNeT is + implemented (currently, sockets, TCP, and IP). This option limits + MAGNeT reporting to only socket-layer activities. + +A Linux system running with MAGNeT installed has two special files: + +/proc/net/magnet - + Displaying this file displays various information about the + currently running MAGNeT version, including size of data + packets, size of the kernel buffer, and other pertinent + details. + +/dev/magnet - + A special device file must be created to perform the mmap()ing from + the kernel to user-level space. /proc/net/magnet contains the + name and major/minor mode numbers this special file requires. + diff -PBbruw linux/include/linux/magnet.h magnet/include/linux/magnet.h --- linux/include/linux/magnet.h Wed Dec 31 17:00:00 1969 +++ magnet/include/linux/magnet.h Wed Sep 26 13:12:30 2001 @@ -0,0 +1,306 @@ +/* + * MAGNET Monitor for Application Generated Network Traffic + * - Define basic data structures for socket instrumentation + * + * Version: @(#)magnet.h 1.0.0 2001/03/06 + * + * Author: Jeff Hay, + * + * Fixes: + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version + * 2 of the License, or (at your option) any later version. + * + */ + +/** +** This software and ancillary information (herein called "SOFTWARE") called +** Monitor for Application-Generated Network Traffic (MAGNeT) +** is made available under the terms described here. The SOFTWARE has been +** approved for release with associated LA-CC number LA-CC 01-32. +** +** Unless otherwise indicated, the SOFTWARE has been authored by an employee or +** emplyees of the University of California, operator of the Los Alamos National +** Laboratory under Contract No. W-7405-ENG-36 with the U.S. Department of Energy. +** The U.S. Government has rights to use, reproduce, and distribute the SOFTWARE. +** The public may copy, distribute, prepare derivative works and publicly display +** this SOFTWARE without charge, provided that this Notice and any statement of +** authorship are reproduced on all copies. Neither the Government nor the +** University makes any warranty, express of implied, or assumes any liability or +** responsibility for the use of this SOFTWARE. +** +** If SOFTWARE is modified to produce derivative works, such modified SOFTWARE +** should be clearly marked, so as not to confuse it with the version available +** from LANL. +**/ + +#ifndef _MAGNET_H +#define _MAGNET_H + +#include + +#ifdef __KERNEL__ +/* for macrology */ +#include +#include +#include +#endif + +#include /* Need PAGE_SIZE definition (for user-mode apps) */ +#define MAGNET_DATA_SIZE 64*PAGE_SIZE /* System-wide magnet data buffer */ + +/* Per-protocol/event storage area buffers */ + +enum { /* Defined magnet event types */ + MAGNET_LOST = 0, /* Lost data. "size" = # records lost */ + + MAGNET_SOCK = 0xF, /* Default - socket operations */ + MAGNET_SOCK_SEND = 0x1, /* Message sent to lower level protocol */ + MAGNET_SOCK_RECV = 0x2, /* Message received from lower level protocol */ + MAGNET_SOCK_SENDCALL = 0x3, /* Application called send() */ + MAGNET_SOCK_RECVCALL = 0x4, /* Application called recv() */ + + MAGNET_TCP = 0xF0, /* TCP Protocol event */ + MAGNET_TCP_SSOCK = 0x10, /* Received message from socket */ + MAGNET_TCP_RSOCK = 0x20, /* Sent message to socket */ + MAGNET_TCP_SIP = 0x30, /* Received message from IP layer */ + MAGNET_TCP_RIP = 0x40, /* Sent message to IP layer */ + + MAGNET_IP = 0xF00, /* IP Protocol event */ + MAGNET_IP_SEND = 0x100, /* Sent message to network device */ + MAGNET_IP_RECV = 0x200, /* Received message from network device */ + + MAGNET_UNKNOWN = 0x1000, /* Unknown Protocol - max value */ + MAGNET_EMPTY = 0x0FEFEFEFE,/* Empty Space created by mkmagnet */ + MAGNET_SYSINFO = 0x01234567, /* System Info packet (endian-aware MAGNeT) */ + MAGNET_OLDSYSINFO = 0x0FFFFFFFF /* Old System Info packet */ +}; +#define MAX_MAGNET_EVENT MAGNET_UNKNOWN + +/* MAGNeT Version number: 0x 00 00 00 00 + Major Minor Revision Size */ +/* The "Size" field, when found in a MAGNET_SYSINFO record, contains the + size in bytes of the MAGNeT instrumentation record at run-time */ +#define MAGNET_VERSION 0x01000200 +/* History: 0x00000000 -> Pre versioning info + 0x01000000 -> MAGNeT 1.0.0 + - CPU Cycle counter in MAGNET_SYSINFO is now in + kHZ, not Hz. + 0x01000100 -> MAGNeT 1.0.1 + - PowerPC port, endian-aware + 0x01000200 -> MAGNeT 1.0.2 + - Cleared LANL distribution tests; public release +*/ + +#ifdef CONFIG_MAGNET_EXTENDED + +/* Data and routines for extended instrumentation. This significantly adds + to the space magnet packets take up in memory, but provides very detailed + reporting about the current running state. Use as needed. + Ideally, this would be a run-time-selectable thing, rather then a build-time + select. + + (data storage sizes in here need to be made platform-generic for both + kernel and user space routines.....) */ + + +/*** +// Extended SOCK-type information +***/ +struct magnet_sock { + pid_t pid; + char name[11]; +}; /* struct magnet_sock */ + +#ifdef __KERNEL__ +extern __inline__ struct magnet_sock magnet_no_data(void) +{ + struct magnet_sock sock; + + memset((void *)&sock, '\0', sizeof(struct magnet_sock)); + if (current == NULL) + { + sock.pid = 0; + return sock; + } + sock.pid = current->pid; + +// This doesn't work. Things just freeze. Not fun. Don't do it. +// if (current->mm != NULL) +// access_process_vm(current, current->mm->arg_start, sock.name, 1, 0); +// (It is supposed to get the PID's command line as an identifier... ah well..) + return sock; +} + +/*** +// Dummy extended type for "No extended information" +***/ +#define MAGNET_NO_DATA (union magnet_ext_data) magnet_no_data() +#endif + +/*** +// Extended TCP-type information +***/ +struct magnet_tcp { /* data from "struct tcp_opt" in include/net/sock.h */ + + unsigned short source; /* TCP source port number */ + unsigned short dest; /* TCP destination port number */ + + unsigned long snd_wnd; /* The window we expect to receive */ + + unsigned long srtt; /* smothed round trip time << 3 */ + unsigned long rto; /* retransmit timeout */ + + unsigned long packets_out; /* Packets which are "in flight" */ + unsigned long retrans_out; /* Retransmitted packets out */ + + unsigned long snd_ssthresh; /* Slow start size threshold */ + unsigned long snd_cwnd; /* Sending congestion window */ + + unsigned long rcv_wnd; /* Current receiver window */ + unsigned long write_seq; /* Tail+1 of data in tcp send buffer */ + unsigned long copied_seq; /* Head of yet unread data */ + + unsigned short fin:1,syn:1,rst:1,psh:1,ack:1,urg:1,ece:1,cwr:1; /* TCP flags*/ +}; /* struct magnet_tcp */ + +#ifdef __KERNEL__ + extern __inline__ struct magnet_tcp magnet_tcp_data( + struct tcp_opt *tp, struct sk_buff *skb) +{ + struct magnet_tcp tcp; + struct tcphdr *head; + + memset((void *)&tcp, '\0', sizeof(struct magnet_tcp)); + if (tp != NULL) { + tcp.snd_wnd = tp->snd_wnd; tcp.srtt = tp->srtt; tcp.rto = tp->rto; + tcp.packets_out = tp->packets_out; tcp.retrans_out = tp->retrans_out; + tcp.snd_ssthresh = tp->snd_ssthresh; tcp.snd_cwnd = tp->snd_cwnd; + tcp.write_seq = tp->write_seq; tcp.copied_seq = tp->copied_seq; + } + +#if 0 + /* Weird bug here -- even after the NULL checks, dereferencing head-> + will consistaintly fail the first time data comes in; works fine + after that.... dunno why... */ + if (skb != NULL) { + head = skb->h.th; + if (head != NULL) { + tcp.source = head->source; + tcp.dest = head->dest; + tcp.fin = head->fin; tcp.syn = head->syn; tcp.psh = head->psh; + tcp.ack = head->ack; tcp.urg = head->urg; tcp.ece = head->ece; + tcp.cwr = head->cwr; + } + } +#endif + + return tcp; +} +#define MAGNET_TCP_DATA (union magnet_ext_data) magnet_tcp_data(tp, skb) +#endif + +/*** +// Extended IP-type information +***/ + +struct magnet_ip { + unsigned char version; + unsigned char tos; + unsigned short id; + unsigned short frag_off; + unsigned char ttl; + unsigned char protocol; +}; /* struct magnet_ip */ + +#ifdef __KERNEL__ +extern __inline__ struct magnet_ip magnet_ip_data(struct iphdr *iph) +{ + struct magnet_ip ip; + + memset((void *)&ip, '\0', sizeof(struct magnet_ip)); + if (iph != NULL) { + ip.version = iph->version; ip.tos = iph->tos; + ip.id = iph->id; ip.frag_off = iph->frag_off; + ip.ttl = iph->ttl; ip.protocol = iph->protocol; + } + return ip; + +} +#define MAGNET_IP_DATA (union magnet_ext_data) magnet_ip_data(skb->nh.iph) +#endif + +/*** +// Declaration of magnet extended data type +***/ + +union magnet_ext_data { /* Stores data for different events */ + struct magnet_sock sock; + struct magnet_tcp tcp; + struct magnet_ip ip; +}; + +#else // not defined CONFIG_MAGNET_EXTENDED + +extern union magnet_ext_data { int dummy; } magnet_false_data; +#define MAGNET_IP_DATA magnet_false_data +#define MAGNET_TCP_DATA magnet_false_data +#define MAGNET_NO_DATA magnet_false_data + +#endif // defined CONFIG_MAGNET_EXTENDED + +struct magnet_data { + void *sockid; + unsigned long long timestamp; /* Time of event (cycle counter) */ + unsigned int event; /* Kind of event */ + int size; /* Size (in bytes) of event */ + union magnet_ext_data data; /* Event-specific data */ +}; /* struct magnet_data */ + + +#ifdef __KERNEL__ + +/* + Data structures and routines to do shared memory with user-space progs +*/ + +#include + +extern int magnet_open(struct inode *inode, struct file *file); +extern int magnet_close(struct inode *inode, struct file *file); +extern int magnet_mmap(struct file *file, struct vm_area_struct *vma); + +#if 0 +extern struct page * magnet_nopage(struct vm_area_struct *vma, + unsigned long address, + int no_share); +#endif + +/* + Routines to access magnet data +*/ + +extern int magnet_procread(char *buf, char **extbuf, off_t offset, int len); + +/* + Routines to access magnet_buffers +*/ + +/* Add an event to a per-socket queue */ +extern __inline__ void magnet_add(struct sock *sk, + unsigned int magnet_event, + int magnet_size, + union magnet_ext_data data); + +/* Initialize this socket's instrumentation interface */ +extern __inline__ int magnet_setup(void); +extern __inline__ void magnet_init(void); + +/* Teardown this socket's instrumentation ineterface */ +extern __inline__ void magnet_destroy(void); + +#endif /* __KERNEL__ */ + +#endif /* _MAGNET_H */ diff -PBbruw linux/net/Config.in magnet/net/Config.in --- linux/net/Config.in Tue Mar 6 23:44:15 2001 +++ magnet/net/Config.in Wed Sep 26 13:08:03 2001 @@ -17,6 +17,12 @@ bool ' Network packet filtering debugging' CONFIG_NETFILTER_DEBUG fi bool 'Socket Filtering' CONFIG_FILTER +if [ "$CONFIG_EXPERIMENTAL" = "y" ]; then + bool 'MAGNeT: Socket Instrumentation interface' CONFIG_MAGNET +fi +if [ "$CONFIG_MAGNET" = "y" ]; then + source net/magnet/Config.in +fi tristate 'Unix domain sockets' CONFIG_UNIX bool 'TCP/IP networking' CONFIG_INET if [ "$CONFIG_INET" = "y" ]; then diff -PBbruw linux/net/Makefile magnet/net/Makefile --- linux/net/Makefile Fri Dec 29 15:07:24 2000 +++ magnet/net/Makefile Wed Sep 26 13:08:03 2001 @@ -45,6 +45,8 @@ subdir-$(CONFIG_DECNET) += decnet subdir-$(CONFIG_ECONET) += econet +subdir-$(CONFIG_MAGNET) += magnet + obj-y := socket.o $(join $(subdir-y), $(patsubst %,/%.o,$(notdir $(subdir-y)))) ifeq ($(CONFIG_NET),y) diff -PBbruw linux/net/README magnet/net/README --- linux/net/README Wed May 16 11:31:27 2001 +++ magnet/net/README Wed Sep 26 13:08:03 2001 @@ -22,5 +22,5 @@ wanrouter gene@compuserve.com, jaspreet@sangoma and dm@sangoma.com unix alan@lxorguk.ukuu.org.uk x25 g4klx@g4klx.demon.co.uk - +magnet radiant-software@lanl.gov diff -PBbruw linux/net/core/sock.c magnet/net/core/sock.c --- linux/net/core/sock.c Wed Apr 25 15:57:39 2001 +++ magnet/net/core/sock.c Wed Sep 26 13:08:03 2001 @@ -589,6 +589,9 @@ sock_lock_init(sk); } + /* MAGNET: This is where we'd call the per-socket magnet_setup() + function, if we had one... */ + return sk; } @@ -597,6 +600,9 @@ #ifdef CONFIG_FILTER struct sk_filter *filter; #endif + + /* MAGNET: This is where we'd call the per-socket magnet_destroy() + function, if we had one... */ if (sk->destruct) sk->destruct(sk); diff -PBbruw linux/net/ipv4/ip_input.c magnet/net/ipv4/ip_input.c --- linux/net/ipv4/ip_input.c Thu Apr 12 13:11:39 2001 +++ magnet/net/ipv4/ip_input.c Wed Sep 26 13:08:03 2001 @@ -144,6 +144,10 @@ #include #include +#ifdef CONFIG_MAGNET +#include +#endif + /* * SNMP management statistics */ @@ -246,6 +250,12 @@ struct sock *raw_sk = raw_v4_htable[hash]; struct inet_protocol *ipprot; int flag; + +#ifdef CONFIG_MAGNET + /* Mark IP hand-off to the upper layer protocol */ + magnet_add(raw_sk, MAGNET_IP_RECV, skb->len, + MAGNET_IP_DATA); +#endif /* If there maybe a raw socket we must check - if not we * don't care less diff -PBbruw linux/net/ipv4/ip_output.c magnet/net/ipv4/ip_output.c --- linux/net/ipv4/ip_output.c Thu Apr 12 13:11:39 2001 +++ magnet/net/ipv4/ip_output.c Wed Sep 26 13:08:03 2001 @@ -77,6 +77,10 @@ #include #include +#ifdef CONFIG_MAGNET +#include +#endif + /* * Shall we try to damage output packets if routing dev changes? */ @@ -149,6 +153,11 @@ ip_options_build(skb, opt, daddr, rt, 0); } ip_send_check(iph); + +#ifdef CONFIG_MAGNET + /* Mark IP hand-off to the physical layer */ + magnet_add(sk, MAGNET_IP_SEND, skb->len, MAGNET_IP_DATA); +#endif /* Send it out. */ return NF_HOOK(PF_INET, NF_IP_LOCAL_OUT, skb, NULL, rt->u.dst.dev, diff -PBbruw linux/net/ipv4/tcp.c magnet/net/ipv4/tcp.c --- linux/net/ipv4/tcp.c Wed May 16 11:31:27 2001 +++ magnet/net/ipv4/tcp.c Wed Sep 26 13:08:03 2001 @@ -258,6 +258,10 @@ #include #include +#if defined (CONFIG_MAGNET) +#include +#endif + int sysctl_tcp_fin_timeout = TCP_FIN_TIMEOUT; struct tcp_mib tcp_statistics[NR_CPUS*2]; @@ -1608,6 +1612,11 @@ } if (!(flags&MSG_TRUNC)) { +#ifdef CONFIG_MAGNET + /* Mark the TCP hand-off to Socket */ + magnet_add(sk,MAGNET_TCP_RSOCK,used,MAGNET_TCP_DATA); +#endif + err = skb_copy_datagram_iovec(skb, offset, msg->msg_iov, used); if (err) { /* Exception. Bailout! */ diff -PBbruw linux/net/ipv4/tcp_ipv4.c magnet/net/ipv4/tcp_ipv4.c --- linux/net/ipv4/tcp_ipv4.c Wed Apr 25 15:57:39 2001 +++ magnet/net/ipv4/tcp_ipv4.c Wed Sep 26 13:08:03 2001 @@ -63,6 +63,10 @@ #include #include +#if defined (CONFIG_MAGNET) +#include +#endif + extern int sysctl_ip_dynaddr; /* Check TCP sequence numbers in ICMP packets. */ @@ -1594,6 +1598,9 @@ struct tcphdr *th; struct sock *sk; int ret; +#ifdef CONFIG_MAGNET + struct tcp_opt *tp = NULL; +#endif if (skb->pkt_type!=PACKET_HOST) goto discard_it; @@ -1633,6 +1640,13 @@ if (!sk) goto no_tcp_socket; + + +#ifdef CONFIG_MAGNET + /* Mark TCP receive from IP layer */ + tp = &(sk->tp_pinfo.af_tcp); + magnet_add(sk, MAGNET_TCP_SIP, skb->len, MAGNET_TCP_DATA); +#endif process: if(!ipsec_sk_policy(sk,skb)) diff -PBbruw linux/net/ipv4/tcp_output.c magnet/net/ipv4/tcp_output.c --- linux/net/ipv4/tcp_output.c Thu Apr 12 13:11:39 2001 +++ magnet/net/ipv4/tcp_output.c Wed Sep 26 13:08:03 2001 @@ -40,6 +40,10 @@ #include +#ifdef CONFIG_MAGNET +#include +#endif + /* People can turn this off for buggy TCP's found in printers etc. */ int sysctl_tcp_retrans_collapse = 1; @@ -274,6 +278,10 @@ TCP_INC_STATS(TcpOutSegs); +#ifdef CONFIG_MAGNET + /* Mark TCP hand-off to the IP layer */ + magnet_add(sk, MAGNET_TCP_RIP, skb->len, MAGNET_TCP_DATA); +#endif err = tp->af_specific->queue_xmit(skb); if (err <= 0) return err; @@ -304,6 +312,11 @@ void tcp_send_skb(struct sock *sk, struct sk_buff *skb, int force_queue, unsigned cur_mss) { struct tcp_opt *tp = &(sk->tp_pinfo.af_tcp); + +#ifdef CONFIG_MAGNET + /* Mark the socket hand-off to TCP */ + magnet_add(sk,MAGNET_TCP_SSOCK,skb->len,MAGNET_TCP_DATA); +#endif /* Advance write_seq and place onto the write_queue. */ tp->write_seq = TCP_SKB_CB(skb)->end_seq; diff -PBbruw linux/net/magnet/Config.in magnet/net/magnet/Config.in --- linux/net/magnet/Config.in Wed Dec 31 17:00:00 1969 +++ magnet/net/magnet/Config.in Wed Sep 26 13:08:03 2001 @@ -0,0 +1,5 @@ +# +# MAGNeT configuration +# +bool ' MAGNeT: Enable Extended Instrumentation' CONFIG_MAGNET_EXTENDED +bool ' MAGNeT: Instrument only socket calls (no protocol stack)' CONFIG_MAGNET_SOCK_ONLY diff -PBbruw linux/net/magnet/Makefile magnet/net/magnet/Makefile --- linux/net/magnet/Makefile Wed Dec 31 17:00:00 1969 +++ magnet/net/magnet/Makefile Wed Sep 26 13:08:03 2001 @@ -0,0 +1,15 @@ +# +# Makefile for the socket instrumentation interface. +# +# Note! Dependencies are done automagically by 'make dep', which also +# removes any old dependencies. DON'T put your own dependencies here +# unless it's something special (ie not a .c file). +# +# Note 2! The CFLAGS definition is now in the main makefile... + +O_TARGET := + +obj-y = magnet.o + +include $(TOPDIR)/Rules.make + diff -PBbruw linux/net/magnet/magnet-parse.c magnet/net/magnet/magnet-parse.c --- linux/net/magnet/magnet-parse.c Wed Dec 31 17:00:00 1969 +++ magnet/net/magnet/magnet-parse.c Wed Sep 26 13:08:03 2001 @@ -0,0 +1,441 @@ +/* + MAGNeT - Monitor for Application-Generated Network Traffic + - MAGNeT data file to ASCII text convertor + - Written by Jeff Hay, radiant-software@lanl.gov +*/ + +/* This is an application, distributed with the kernel source. Therefore, its + likely to be built with the default kernel Makefiles, which define the + __KERNEL__ flag. Since we don't really want that, undefine it for us. */ +#ifdef __KERNEL__ +#undef __KERNEL__ +#endif + +#include +#include +#include + +/* Performs big<->little endian translation of a generic buffer */ +void eswap(unsigned char *buffer, int Length) { + int Count, Temp; + + for (Count = 0; Count < (Length >> 1); Count++) { + Temp = (buffer)[Length - Count - 1] & 0x0FF; + (buffer)[Length - Count - 1] = (buffer)[Count] & 0x0FF; + (buffer)[Count] = Temp & 0x0FF; + } + + return; +} /* eswap() */ + +/* Performs big<->little endian translation of int-sized values */ +int eswap_int(int value) { + + eswap((char *)&value, sizeof(int)); + + return value; +} /* eswap_int() */ + +/* Performs big<->little endian translation of longlong-sized values */ +long long eswap_longlong(long long value) { + + eswap((unsigned char *)&value, sizeof(long long)); + + return value; +} /* eswap_longlong() */ + + +/* Returns a string representation of a MAGNeT event name */ +const char *magnet_name(int event) { + const char *name; + + switch (event) { + case MAGNET_LOST: name = "Lost"; + break; + + case MAGNET_SOCK: name = "Socket"; + break; + case MAGNET_SOCK_SEND: name = "Send"; + break; + case MAGNET_SOCK_RECV: name = "Receive"; + break; + case MAGNET_SOCK_SENDCALL: name = "SndCall"; + break; + case MAGNET_SOCK_RECVCALL: name = "RcvCall"; + break; + + case MAGNET_TCP: name = "TCP"; + break; + case MAGNET_TCP_SSOCK: name = "Ap->TCP"; + break; + case MAGNET_TCP_RSOCK: name = "TCP->Ap"; + break; + case MAGNET_TCP_SIP: name = "IP->TCP"; + break; + case MAGNET_TCP_RIP: name = "TCP->IP"; + break; + + case MAGNET_IP: name = "IP"; + break; + case MAGNET_IP_SEND: name = "IP->net"; + break; + case MAGNET_IP_RECV: name = "net->IP"; + break; + + case MAGNET_EMPTY: name = "Empty"; + break; + + case MAGNET_SYSINFO: name = "SysInfo"; + break; + + case MAGNET_OLDSYSINFO: name = "SysInfo (Non-Endian Aware)"; + break; + + default: + name = "Unknown"; + } + + + return name; +} /* magnet_name() */ + +/* Processes MAGNeT protocol-specific extra data to ASCII text. + LIMITATIONS: + o Only works for machines with a native endian format + identical to the data file format + o Only compiled in if CONFIG_MAGNET_EXTENDED is enabled + in the current kernel build. +*/ +char * magnet_extra_data(unsigned int extra[], int size, int event, + int endianswap, int padding) { + static char databuf[1024]; + int count; + union magnet_ext_data data; + + if ((size <= 0) || (size > sizeof(data))) + return ""; + + /* Copy data into MAGNeT extra data structure */ + memcpy(&data, extra, (sizeof(unsigned int) * size)); + count = 0; + +#ifdef CONFIG_MAGNET_EXTENDED + if (event & MAGNET_SOCK) { + count += sprintf(databuf, "pid=%d,name=%s", + data.sock.pid, data.sock.name); + /* Socket Events */ + } else if (event & MAGNET_TCP) { + /* TCP Events */ + count += sprintf(databuf, +#if 0 + "source=%u,dest=%u," + "fin=%u,rst=%u,psh=%u,ack=%u,urg=%u,ece=%u," + "cwr=%u" +#endif + "snd_wnd=%lu,srtt=%lu,rto=%lu,packets_out=%lu" + "retrans_out=%lu,snd_ssthresh=%lu," + "snd_cwnd=%lu,rcv_wnd=%lu,write_seq=%lu," + "copied_seq=%lu", +#if 0 + data.tcp.source,data.tcp.dest, + data.tcp.fin,data.tcp.rst,data.tcp.psh, + data.tcp.ack,data.tcp.urg,data.tcp.ece, + data.tcp.cwr, +#endif + data.tcp.snd_wnd,data.tcp.srtt,data.tcp.rto, + data.tcp.packets_out,data.tcp.retrans_out, + data.tcp.snd_ssthresh,data.tcp.snd_cwnd, + data.tcp.rcv_wnd,data.tcp.write_seq, + data.tcp.copied_seq); + } else if (event & MAGNET_IP) { + /* IP Events */ + count += sprintf(databuf, + "version=%u,tos=%u,id=%u,frag_off=%u," + "ttl=%u,protocol=%u", + data.ip.version,data.ip.tos, data.ip.id, + data.ip.frag_off,data.ip.ttl, + data.ip.protocol); + } +#endif // defined CONFIG_MAGNET_EXTENDED + + databuf[count] = '\0'; + return databuf; +} /* magnet_extra_data() */ + +/* Reads and displays a single record from "infile", using the given + Record Size, Endian Operations, and Padding, and other specifiers. + "CPU" specifies record CPU speed in kHZ (from MAGNET_SYSINFO record). + Returns 0 on success, <0 if error reading file, >0 if unparsable data. */ +int parse_records(FILE *infile, int recordsize, int endianswap, int padding, + int CPU, unsigned long long starttime) { + unsigned int extra[256]; + unsigned int sockid, event, size, junk; + unsigned long long timestamp; + int count, skip, index; + + /* Read in all appropriate information */ + if (fread(&sockid, sizeof(sockid), 1, infile) < 1) + return -1; + for (count = 0; count < (padding / sizeof(junk)); count++) + if (fread(&junk, sizeof(junk), 1, infile) < 1) + return -1; + if ((fread(×tamp, sizeof(timestamp), 1, infile) < 1) || + (fread(&event, sizeof(event), 1, infile) < 1) || + (fread(&size, sizeof(size), 1, infile) < 1)) + return -1; + + /* Read in extra data (remainder of record) */ + skip = recordsize - sizeof(sockid) - sizeof(event) - sizeof(size) - + sizeof(timestamp) - padding; + index = 0; + for (count = 0; count < (skip / sizeof(junk)); count++) { + if (fread(&junk, sizeof(junk), 1, infile) < 1) + return -1; + extra[count++] = junk; + } + + /* Perform endian translation, if needed */ + if (endianswap) { + sockid = eswap_int(sockid); + size = eswap_int(size); + event = eswap_int(event); + timestamp = eswap_longlong(timestamp); + } + + if (event == MAGNET_EMPTY) { + fprintf(stderr, "Found Empty Record; Stopping Processing\n"); + return 1; + } else { + printf("%llu\t%08x\t%6d\t%s\t%s\n", timestamp, sockid, size, + magnet_name(event), + magnet_extra_data(extra,skip,event,endianswap,padding)); + } + + return 0; +} /* parse_records() */ + +/* Reads "infile" and parses it, assuming a MAGNeT data stream. Outputs to + STDOUT. Returns 0 on success; < 0 if error occured reading file; > 0 if + data stream appears to be non-MAGNeT. "ovr_*" parameters allow overriding + of automatically-determined MAGNeT data configurations. */ +int do_magnet_parse(FILE *infile, int verbose, + int ovr_ver, int ovr_rec, int ovr_end) { + int endianswap; + unsigned int data1, data2, padding; + unsigned long long starttime; + unsigned long mask; + unsigned int cpu; + int count, skip; + int recordsize, magnet_ver; + unsigned long long bytecount; + unsigned long recordcount; + float cpu_speed; + + /* First byte will contain version/record size info; store it + until we know how to process it. */ + if (fread(&data1, sizeof(data1), 1, infile) < 1) + return -1; + /* Look for a MAGNET_SYSINFO byte, in whatever endian format. If this + byte isn't in the next 6 bytes, give up -- if it is, continue + processing based on what we learn from the byte. As we go, store + read information in "starttime"(we're skipping over the timestamp)*/ + endianswap = -1; + count = 0; + starttime = 0; + while ((count < 5) && (endianswap < 0)) { + count++; + if (fread(&data2, sizeof(data2), 1, infile) < 1) + return -1; + if (data2 == MAGNET_SYSINFO) + endianswap = 0; + else if (eswap_int(data2) == MAGNET_SYSINFO) + endianswap = 1; + else if (data2 == MAGNET_OLDSYSINFO) + endianswap = 2; + else + starttime = ((starttime << (sizeof(data2) * 8)) | data2); + } + + mask = 1; + if ((*(char *)&mask) == 1) { + /* If we're running on a little-endian machine, we've just + built starttime with its composite words swapped. Fixing..*/ + mask = (unsigned long)-1; + starttime = ((starttime & mask) << (sizeof(mask)*8)) | + ((starttime & ((unsigned long long)mask << (sizeof(mask)*8))) >> + (sizeof(mask) * 8)); + } + + if (count == 5) + return 1; /* Doesn't appear to be a MAGNeT data stream; abort */ + + /* Determine bytes of padding between "sockid" and "timestamp" */ + padding = (((count-1)*sizeof(data2)) - sizeof(starttime)); + + if (endianswap == 2) { + fprintf(stderr, "Warning: Non-Endian Aware Data\n"); + /* Convert old MAGNeT version number to new format */ + data1 = ((data1 & 0x0FFFF0000) | ((data1 & 0xFF) << 8) | + sizeof(struct magnet_data)); + if (ovr_end > 0) /* Make it conform to expected endian order */ + data1 = eswap_int(data1); + endianswap = 0; + } + if (ovr_end > 0) { + fprintf(stderr, "Warning: Using Specified Endian Override\n"); + endianswap = ovr_end; + } + + if (endianswap) + data1 = eswap_int(data1); + recordsize = ((data1) & 0x0FF); + magnet_ver = ((data1) & 0x0FFFFFF00); + fprintf(stderr, + "Data Stream is MAGNeT version %d.%d.%d with %d-byte records\n", + (magnet_ver & 0x0FF000000) >> 24,(magnet_ver & 0x0FF0000) >> 16, + (magnet_ver & 0x0FF00) >> 8, recordsize); + if ((ovr_ver > 0) | (ovr_rec > 0)) { + if (ovr_ver > 0) { magnet_ver = ovr_ver; } + if (ovr_rec > 0) { recordsize = ovr_rec; } + fprintf(stderr, + "Overriding to MAGNeT version %d.%d.%d with %d-byte records\n", + (magnet_ver & 0x0FF000000) >> 24,(magnet_ver & 0x0FF0000) >> 16, + (magnet_ver & 0x0FF00) >> 8, recordsize); + } + if (endianswap) { + starttime = eswap_longlong(starttime); + } + + if (verbose) + fprintf(stderr, "Trace stated at %llu cycles\n", starttime); + + if (recordsize < 1) { + fprintf(stderr, "Error: Invalid Record Size; aborting\n"); + return -1; + } + + /* Read in CPU speed ("size" field) */ + if (fread(&data2, sizeof(data2), 1, infile) < 1) + return -1; + if (endianswap) + cpu = eswap_int(data2); + else + cpu = data2; + + cpu_speed = cpu; + if (cpu < 1000) + { cpu_speed = cpu; /* CPU reported in MHz (pre-release MAGNeT) */ } + else + { cpu_speed = cpu / 1000.0; /* CPU Speed reported in KHz */ } + + /* Note: An apparent bug in the PPC version of gcc seems to make + cpu_speed always be darn near zero.... Oh well.... */ + if (verbose) + fprintf(stderr, "Collection CPU speed: %4.0f MHz\n",cpu_speed); + + /* Skip over remaining data in this record */ + count = sizeof(data1) + padding + sizeof(starttime) + sizeof(data2) + + sizeof(cpu); + skip = recordsize - count; + for (count = 0; count < (skip / sizeof(data1)); count++) + if (fread(&data1, sizeof(data1), 1, infile) < 1) + return -1; + + recordcount = 0; + bytecount = 0; + while (parse_records(infile, recordsize, endianswap, padding, + cpu, starttime) == 0) { + recordcount++; + bytecount += recordsize; + } + fprintf(stderr, "Read total of %llu bytes; %lu records\n", + bytecount, recordcount); + + return 0; +} /* do_magnet_parse() */ + +int main(int argc, char *argv[]) { + FILE *infile; + int error, argcount; + int version, record, endian; + + fprintf(stderr, "%s: MAGNeT Data File Reader\n\n", argv[0]); + + if (argc < 2) { + fprintf(stderr," syntax: %s [opts]\n\n", + argv[0]); + fprintf(stderr, "[opts] consists of:\n"); + fprintf(stderr,"\t-r \tUse record size of bytes\n"); + fprintf(stderr,"\t-v \tAssume data is MAGNeT version:\n"); + fprintf(stderr, "\t\t\t = 0, version = %d.%d.%d\n", + (MAGNET_VERSION & 0x0FF000000) >> 24, + (MAGNET_VERSION & 0x0FF0000) >> 16, + (MAGNET_VERSION & 0x0FF00) >> 8); + fprintf(stderr, "\t\t\t = 65536, version = 1.0.0\n"); + fprintf(stderr, "\t\t\t = 65537, version = 1.0.1\n"); + fprintf(stderr,"\t-e \tAssume data is in:\n"); + fprintf(stderr,"\t\t\t = 0, native endian format\n"); + fprintf(stderr,"\t\t\t = 1, non-native endian format\n"); + fprintf(stderr, "All options are automatically determined from the data if not specified.\n"); + fprintf(stderr, " may be \"-\" for STDIN.\n"); + fprintf(stderr, "NOTE: MAGNeT extended data is *not* endian processed!\n"); + return -1; + } + + /* Attempt to open input file (or stdin, if requested) */ + if (!strcmp(argv[1], "-")) { + if ((infile = fdopen(0, "r")) == NULL) { + fprintf(stderr, "%s: Unable to read stdin\n", argv[0]); + return -2; + } + } else { + if ((infile = fopen(argv[1], "r")) == NULL) { + fprintf(stderr, "%s: Unable to read input file \"%s\"\n", + argv[0], argv[1]); + return -2; + } + } + + /* Process other command-line options */ + + argcount = 2; + version = record = endian = 0; + while (argcount < argc) { + if (!strcmp(argv[argcount], "-v")) { + if (argc <= ++argcount) { + fprintf(stderr,"%s: Please supply version number.\n", argv[0]); + return -1; + } + version = atoi(argv[argcount]) << 8; + if (version == 0) { version = MAGNET_VERSION; } + } + if (!strcmp(argv[argcount], "-r")) { + if (argc <= ++argcount) { + fprintf(stderr,"%s: Please supply record size.\n", argv[0]); + return -1; + } + record = atoi(argv[argcount]); + if (record == 0) { record = sizeof(struct magnet_data); } + } + if (!strcmp(argv[argcount], "-e")) { + if (argc <= ++argcount) { + fprintf(stderr, "%s: Please supply endian format.\n", argv[0]); + return -1; + } + endian = atoi(argv[argcount]); + } + argcount++; + } + + if ((error = do_magnet_parse(infile, 1, version, record, endian)) > 0) + { + fprintf(stderr, "%s: Unknown Data Stream Format.\n", argv[0]); + return error; + } else if (error < 0) { + fprintf(stderr, "%s: Error Encounted in MAGNeT Data.\n", argv[0]); + return error; + } + + fclose(infile); + return 0; +} /* main() */ diff -PBbruw linux/net/magnet/magnet-read.c magnet/net/magnet/magnet-read.c --- linux/net/magnet/magnet-read.c Wed Dec 31 17:00:00 1969 +++ magnet/net/magnet/magnet-read.c Wed Sep 26 13:08:03 2001 @@ -0,0 +1,235 @@ +/* + MAGNeT - Monitor for Application-Generated Network Traffic + - User-mode monitor information reader + - Written Nov 2000, Jeff Hay, radiant-software@lanl.gov +*/ + +/* This is an application, distributed with the kernel source. Therefore, its + likely to be built with the default kernel Makefiles, which define the + __KERNEL__ flag. Since we don't really want that, undefine it for us. */ +#ifdef __KERNEL__ +#undef __KERNEL__ +#endif + +#include +#include + +#include +#include +#include +#include +#include +#include +#include + +/* Global (ie, syrnconization, etc) variables */ +char *fname; +int fileout; +int magnet, sleepwatch; +float sleepchange; +unsigned int written; +struct magnet_data *kernel_data; +struct magnet_data *save_data; +struct stat outstat; + +struct sigaction act, oact_term, oact_int; + +/* Code to handle TERM signal correctly */ + + +/* Main program code */ + +/* Computes a default time to sleep (in ms) between reading instrumentation + data from the kernel, along with high- and low-water marks (used for dynamic + adjustment of sleep time, if requested) */ +unsigned int get_sleeptime(unsigned int *high_water, + unsigned int *low_water) +{ + unsigned int num_events, packet_size, net_speed, pkts_second; + unsigned int data_rate, drain_rate, sleep_time; + + + /* Initial Sleep time calculation */ + num_events = ((int)(MAGNET_DATA_SIZE)) / sizeof(struct magnet_data); + packet_size = 1; /* Assume worst case => 1 byte packets */ + net_speed = 1000; + pkts_second = (net_speed * 10000000) / (packet_size * 8); + data_rate = (pkts_second / 8); /* 4 events per network packet */ + drain_rate = data_rate / num_events; + + printf(" Kernel Buffer: %d events\n", num_events); + printf("Max network rate: %d events/second.\n", data_rate); + printf("Kernel buffer will be drained %d times per second\n", + drain_rate); + if (sleepchange > 0.0) { + drain_rate = (unsigned int)((float)drain_rate * sleepchange); + printf("At user request, kernel will be dreained %d times per second\n", + drain_rate); + } + sleep_time = 1000000 / drain_rate; + printf("\n"); + + if ((high_water != NULL) && (low_water != NULL) && (sleepwatch)) { + /* high and low water marks */ + *low_water = num_events / 4; + *high_water = 3 * *low_water; + printf("High-water mark: %d events\n", *high_water); + printf("Low-Water mark: %d events\n", *low_water); + } + + return sleep_time; +} /* get_sleeptime() */ + +void read_kernel_data(unsigned int maxwrite) { + unsigned int num_packets, sleep_time, high_water, low_water; + struct magnet_data *out_data; + void *current; + int count, sleepcount; + struct timeval timeout; + int readcount = 0; + + printf("Memory area successfully mapped\n"); + sleep_time = get_sleeptime(&high_water, &low_water); + printf("Recording started; Reading maximum of %d bytes\n", maxwrite); + + num_packets = ((int)(MAGNET_DATA_SIZE)) / sizeof(struct magnet_data); + + count = 0; + current = kernel_data; + written = 0; + out_data = save_data; + while ((maxwrite <= 0) || (written < maxwrite)) { + timeout.tv_sec = 0; + timeout.tv_usec = sleep_time; + select(0, NULL, NULL, NULL, &timeout); + + sleepcount = 0; + + while (((struct magnet_data *)current)->timestamp != 0) { + memcpy(out_data++, current, + sizeof(struct magnet_data)); + written += sizeof(struct magnet_data); + + readcount = 0; + ((struct magnet_data *)current)->timestamp = 0; + current += sizeof(struct magnet_data); + if (++count >= num_packets) { + current = kernel_data; + count = 0; + } + sleepcount++; + } + + if ((sleepwatch) && (sleepcount > 0)) { + if (sleepcount >= high_water) + sleep_time /= 2; + else if (sleepcount <= low_water) + sleep_time *= 2; + } + } + + return; +} /* read_kernel_data() */ + +void terminate(int signal) +{ + printf("Closing shared memory\n"); + + munmap(kernel_data, MAGNET_DATA_SIZE); + close(magnet); + + printf("syncing mmap file\n"); + msync(save_data, written, MS_SYNC); + munmap(save_data, outstat.st_size); + munlock(save_data, outstat.st_size); + close(fileout); + + printf("Saved %u bytes of instrumentation\n", written); + + if (written < outstat.st_size) + truncate(fname, written); + + if (sigaction(SIGTERM, &oact_term, NULL) < 0) + fprintf(stderr, "error: Failed to restore SIGTERM signal handler\n"); + if (sigaction(SIGINT, &oact_int, NULL) < 0) + fprintf(stderr, "error: Failed to restore SIGINT signal handler\n"); + + return; +} + +int main(int argc, char *argv[]) { + int lastarg; + + printf("%s: Monitor for Application Generated Network Traffic\n\n", argv[0]); + + if (argc < 3) { + fprintf(stderr, " syntax: %s [-d] [-c ]\n\n", + argv[0]); + fprintf(stderr, " \"-d\" = Dynamic Sleep Time Adjustment enabled\n"); + fprintf(stderr, " \"-c\" = Multiply Default Sleep Time by \n"); + return -1; + } + + sleepwatch = 0; + if ((argc > 3) && (!strcmp(argv[3], "-d"))) { + sleepwatch = 1; + lastarg = 4; + } else { lastarg = 3; } + if ((argc > lastarg) && (!strcmp(argv[lastarg], "-c"))) { + sleepchange = atof(argv[lastarg+1]); + } else { sleepchange = 0.0; } + + fname = argv[2]; + if ((fileout = open(argv[2], O_RDWR|O_NDELAY)) <= 0) { + fprintf(stderr, "%s: Can not open output file \"%s\"\n", + argv[0], argv[2]); + fprintf(stderr, "%s: Is the file created? If not, try mkmagnet\n", + argv[0]); + return -2; + } + if (fstat(fileout, &outstat) != 0) { + fprintf(stderr, "%s: Can not stat output file \"%s\"\n", + argv[0], argv[2]); + return -2; + } + if (outstat.st_size <= 0) { + fprintf(stderr, "%s: Output file must be non-zero length\n", + argv[0]); + return -3; + } + + if ((magnet = open(argv[1], O_RDONLY)) == -1) { + fprintf(stderr, "%s: \"%s\": ", argv[0], argv[1]); + perror("Invalid MAGNeT device"); + return -1; + } + printf("\tRecording %d bytes of data\n", (int)outstat.st_size); + + /* Set up SIGTERM handler */ + act.sa_handler = &terminate; + sigemptyset(&act.sa_mask); + act.sa_flags = 0; + if (sigaction(SIGTERM, &act, &oact_term) < 0) + fprintf(stderr, "%s: Failed to install SIGTERM signal handler\n", argv[0]); + if (sigaction(SIGINT, &act, &oact_int) < 0) + fprintf(stderr, "%s: Failed to install SIGINT signal handler\n", argv[0]); + + /* Map the shared memory region into our own space */ + if ((int)(kernel_data = (struct magnet_data *) + mmap(NULL,MAGNET_DATA_SIZE,PROT_READ,MAP_PRIVATE,magnet,0)) == -1) { + perror("Can not map kernel instrumentation data"); + return -1; + } + + if ((int)(save_data = (struct magnet_data *) + mmap(NULL,outstat.st_size,PROT_WRITE,MAP_SHARED,fileout,0)) == -1) { + perror("Can not mmap output file"); + return -1; + } + mlock(save_data, outstat.st_size); + + read_kernel_data(outstat.st_size); + + terminate(SIGTERM); + exit(0); +} /* main() */ diff -PBbruw linux/net/magnet/magnet.c magnet/net/magnet/magnet.c --- linux/net/magnet/magnet.c Wed Dec 31 17:00:00 1969 +++ magnet/net/magnet/magnet.c Wed Sep 26 13:09:24 2001 @@ -0,0 +1,322 @@ +/* + * MAGNET Monitor for Application Generated Network Traffic + * + * Version: @(#)magnet.c 1.0.0 2001/03/06 + * + * Author: Jeff Hay, + * + * Fixes: + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version + * 2 of the License, or (at your option) any later version. + */ + +/** +** This software and ancillary information (herein called "SOFTWARE") called +** Monitor for Application-Generated Network Traffic (MAGNeT) +** is made available under the terms described here. The SOFTWARE has been +** approved for release with associated LA-CC number LA-CC 01-32. +** +** Unless otherwise indicated, the SOFTWARE has been authored by an employee or +** emplyees of the University of California, operator of the Los Alamos National +** Laboratory under Contract No. W-7405-ENG-36 with the U.S. Department of Energy. +** The U.S. Government has rights to use, reproduce, and distribute the SOFTWARE. +** The public may copy, distribute, prepare derivative works and publicly display +** this SOFTWARE without charge, provided that this Notice and any statement of +** authorship are reproduced on all copies. Neither the Government nor the +** University makes any warranty, express of implied, or assumes any liability or +** responsibility for the use of this SOFTWARE. +** +** If SOFTWARE is modified to produce derivative works, such modified SOFTWARE +** should be clearly marked, so as not to confuse it with the version available +** from LANL. +**/ + +#include + +#include +#include +#include + +#include +#include + +#include +#include +#include +#include +#include +#include + +#include + +#include + +/* What we need to determine CPU speed (see magnet_cpu_speed() below) */ +#ifdef __powerpc__ +#include +#else +extern unsigned long cpu_khz; +#endif + +/* + mmap() structures to facilitate user/kernel shared memory +*/ + +static struct file_operations magnet_fops = { + mmap: magnet_mmap, + open: magnet_open, + release: magnet_close +}; + +// This corresponds with a /dev/magnet device with +// major number 10, minor MISC_DYNAMIC_MINOR (normally 63) +static struct miscdevice magnet_device = { + minor: MISC_DYNAMIC_MINOR, + name: "magnet", + fops: &magnet_fops +}; + +/* For reporting system info; this is i386-specific */ +// extern unsigned long cpu_khz; + +/* + Globbal Magnet Variables +*/ + +/* Global socket instrumentation variables */ +char *magnet_data; /* kernel-space magnet buffer */ +char *magnet_data_offs; /* where kernelspace is in buffer */ +unsigned int magnet_lost_pkts; /* Info overwritten before user-level read */ +#ifndef CONFIG_MAGNET_EXTENDED +union magnet_ext_data magnet_false_data; +#endif + +/* + Routines to access magnet data +*/ +extern int magnet_procread(char *buf, char **extbuf, off_t offset, int len) +{ + len = 0; + len = sprintf(buf+len, "MAGNeT: Monitor for Application Generated Network Traffic\n"); + len += sprintf(buf+len, "Research tool created by Los Alamos National Laboratory's RADIANT Team\n"); + len += sprintf(buf+len, "(http://www.lanl.gov/radiant)\n\n"); + len += sprintf(buf+len, " MAGNeT Version: %d.%d.%d\n", + (MAGNET_VERSION & 0x0FF000000) >> 24, + (MAGNET_VERSION & 0x0FF0000) >> 16, + (MAGNET_VERSION & 0x0FF00) >> 8); + len += sprintf(buf+len, " MAGNeT Device Name: /dev/%s\n", magnet_device.name); + len += sprintf(buf+len, " MAGNeT Device Number: 10, %d\n", magnet_device.minor); + len += sprintf(buf+len, " Kernel Memory Reserved: %ld Kb (%ld pages)\n", + (MAGNET_DATA_SIZE / 1024), (MAGNET_DATA_SIZE / PAGE_SIZE)); + len += sprintf(buf+len, " Data packet size: %d bytes\n",sizeof(struct magnet_data)); + len += sprintf(buf+len, "Extended Instrumentation: "); +#ifdef CONFIG_MAGNET_EXTENDED + len += sprintf(buf+len, "on\n"); +#else + len += sprintf(buf+len, "off\n"); +#endif + len += sprintf(buf+len, " Instrument All Events: "); +#ifdef CONFIG_MAGNET_SOCK_ONLY + len += sprintf(buf+len, "off\n"); +#else + len += sprintf(buf+len, "on\n"); +#endif + len += sprintf(buf+len, "\n"); + return len; +} + +__inline__ void magnet_add(struct sock *sk, + unsigned int magnet_event, + int magnet_size, + union magnet_ext_data data) +{ + unsigned int datasize = sizeof(struct magnet_data); + struct magnet_data *sicurrent; + unsigned long long currenttime; + + currenttime = get_cycles(); + + if (magnet_data == NULL) + return; + +#ifdef CONFIG_MAGNET_SOCK_ONLY + if ((magnet_event & MAGNET_SOCK) == 0) /*mask out all non-sock calls*/ + return; +#endif + try_magnet_save: + sicurrent = (struct magnet_data *)magnet_data_offs; + if (sicurrent->timestamp == 0) { + if (magnet_lost_pkts > 0) { + sicurrent->sockid = NULL; + sicurrent->event = MAGNET_LOST; + sicurrent->size = magnet_lost_pkts; + } else { + sicurrent->sockid = sk; + sicurrent->event = magnet_event; + sicurrent->size = magnet_size; +#ifdef CONFIG_MAGNET_EXTENDED + sicurrent->data = data; +#endif + } + /* Write timestamp last as a signal to user-space program */ + sicurrent->timestamp = currenttime; + + magnet_data_offs += datasize; + if (magnet_data_offs >= + (magnet_data + MAGNET_DATA_SIZE - datasize - 1)) { + magnet_data_offs = magnet_data; + } + if (magnet_lost_pkts > 0) { + magnet_lost_pkts = 0; + goto try_magnet_save; + } + } else { + magnet_lost_pkts++; + } + + return; +} + + +/* Initialize socket instrumentation interface */ +int magnet_setup() +{ + struct page *page; + int error; + + /* Initialize kernel-space pointer into buffer */ + magnet_data_offs = magnet_data; + + /* Create /proc/net/magnet file for reporting misc. status */ + proc_net_create("magnet", S_IFREG | S_IRUGO, magnet_procread); + + /* Pin down our buffer's pages, so it can't be swapped out */ + for (page = virt_to_page(magnet_data); + page < virt_to_page(magnet_data + MAGNET_DATA_SIZE); + page++) + mem_map_reserve(page); + + /* Register as a misc char device for doing mmap's */ + error = misc_register(&magnet_device); + + return error; +} + +/* Called at boot-time when socket code is being initialized */ +void magnet_init() +{ + printk(KERN_INFO "NET4: MAGNeT: Socket Instrumentation Interface for NET4.0\n"); + if ((magnet_data = (char *)__get_free_pages( + GFP_ATOMIC, get_order(MAGNET_DATA_SIZE))) == NULL) + printk(KERN_ERR "NET4: MAGNeT: No memory for instrumentation buffer\n"); + else + magnet_setup(); + return; +} + +/* Teardown socket instrumentation interface + (this is never called in the current implementation. Code listed here + to make converting this to a module "easy"....) +*/ +void magnet_destroy() +{ + struct page *page; + + /* disable mmap()ing of buffer space */ + misc_deregister(&magnet_device); + + /* un-pin buffer space */ + for (page = virt_to_page(magnet_data); + page < virt_to_page(magnet_data + MAGNET_DATA_SIZE); + page++) + mem_map_unreserve(page); + + /* remove /proc information hook */ + proc_net_remove("magnet"); + return; +} + +/* + mmap() shared-memory routines +*/ + +int magnet_open(struct inode *inode, struct file *file) +{ + /* Do nothing; this is just a dummy procedure to make the + VFS happy... */ +// printk(KERN_INFO "MAGNeT: opening mmap file\n"); + return 0; +} + +int magnet_close(struct inode *inode, struct file *file) +{ + /* Do nothing; this is just a dummy procedure to make the + VFS happy... */ +// printk(KERN_INFO "MAGNeT: closing mmap file\n"); + return 0; +} + +/* Determines the CPU speed; done here so we have a common place to extract + this information on any platform (since the code in the /arch tree doesn't + do a good job of abstracting this information for us). + + Returns speed in Kcycles/second. + + This is the only part of MAGNeT that is really platform dependent, as there + is no common way to get this info across all Linux platforms. However, + /arch/(platform)/kernel/setup.c usually contains the necessary steps.... + (for displaying the CPU speed in /proc/cpuinfo) */ +unsigned int magnet_cpu_speed() +{ +#ifdef __powerpc__ + int *clock; + + clock=(int *)get_property(find_type_devices("cpu"), "clock-frequency", NULL); + + return (*clock / 1000); +#else + /* Don't know the architecture; assume i386 family... */ + return cpu_khz; +#endif +} + +int magnet_mmap(struct file *file, struct vm_area_struct *vma) +{ + /* Set up the shared memory to user-level process */ + unsigned long size = (unsigned long)(vma->vm_end - vma->vm_start); + + /* First, make sure we're within bounds */ + if (size > MAGNET_DATA_SIZE) + return -EINVAL; + + lock_kernel(); + /* Step through our buffer a page at a time, and remap each page + into the user-mode address space */ + if (remap_page_range(vma->vm_start, + virt_to_phys(magnet_data), + size, PAGE_SHARED)) { + printk(KERN_ERR "MAGNeT: remapping failed\n"); + return -EAGAIN; + } + + /* Don't swap this VMA out! */ + vma->vm_flags |= VM_SHM | VM_LOCKED | VM_IO; + + /* Clear the memory so user doesn't read old information */ + memset(magnet_data, 0, size); + magnet_data_offs = magnet_data; + magnet_lost_pkts = 0; + + unlock_kernel(); + + /* Add a system information field to the beginning of the dump */ + magnet_add((void *)(MAGNET_VERSION | sizeof(struct magnet_data)), + MAGNET_SYSINFO, magnet_cpu_speed(), MAGNET_NO_DATA); + + + /* Success! */ + return 0; +} diff -PBbruw linux/net/magnet/magnet.copy magnet/net/magnet/magnet.copy --- linux/net/magnet/magnet.copy Wed Dec 31 17:00:00 1969 +++ magnet/net/magnet/magnet.copy Wed Sep 26 13:08:03 2001 @@ -0,0 +1,39 @@ +#!/bin/sh +# +# magnet.copy +# Sample MAGNeT archiving script. Copies any gzipped files in the +# directory given as the first parameter to a directory on a remote +# machine, using scp (or any other rcp-like command). +### +# Configurable parameters: + +# Host to archive to +ARCH_HOST=server + +# Directory to archive to +ARCH_DIR=/data/magnet/thishost + +# RCP-like program +RCP=/usr/local/bin/scp +RCP_OPTS="-qp" + +# Archive log file +ARCH_FILE=/var/log/magnet + +#################################################################### + +if [ "$1" = "" -o ! -d "$1" ]; then + echo "$0: Invalid directory to archive: syntax \"$0 \"" + exit 1 +fi + +for FILE in `ls $1/*.gz` +do + echo -n `date`": " >> $ARCH_FILE + echo "Archiving $FILE to $ARCH_HOST" >> $ARCH_FILE + $RCP $RCP_OPTS $FILE $ARCH_HOST:$ARCH_DIR/ >> $ARCH_FILE 2>&1 + if [ $? -eq 0 ]; then + rm -f $FILE + fi +done + diff -PBbruw linux/net/magnet/magnet.cron magnet/net/magnet/magnet.cron --- linux/net/magnet/magnet.cron Wed Dec 31 17:00:00 1969 +++ magnet/net/magnet/magnet.cron Wed Sep 26 13:08:03 2001 @@ -0,0 +1,123 @@ +#!/bin/sh +# +# magnet.cron +# This is a wrapper script for the MAGNeT tools. It is +# primairly intended to be run from cron, or upon bootup. The +# script: +# o Ensures that the kernel is running MAGNeT code +# + Aborts gracefully if non-MAGNeTized +# o Ensures that the proper MAGNeT device file exists +# + Creates the file if it does not exist +# + Modifies device numbers if necessary +# o Compresses any un-compressed MAGNeT data +# o Calls the Archive script to archive the data +# o Generates a new MAGNeT data storage file +# o Runs magnet-read with proper parameters +# (as a background process) +# +# Configurable parameters for the script follow. Script must +# be run suid root. The following programs must be in the +# current path when the script is run: +# +# ls echo date dc cut rm mknod kill awk ps hostname gzip +# +# If the first parameter to this script is "-n", the script will +# echo the commands it would normally execute to stdout, and will +# NOT execute them. +################################################################# +# Set up the following constants to conform to your system +# - Path names should NOT end with a "/" + +# Complete path to magnet binary files +MAGNET_BIN="/bin/magnet" + +# Complete path to magnet data storage area +MAGNET_DATA="/data/magnet" + +# Size of MAGNeT data storage files, in Megabytes +MAGNET_SAVESIZE=100 + +# Parameters to pass to "magnet-read" +MAGNET_OPTS="" + +# MAGNeT archiving script name +# This script is called to move all gzipped files in $MAGNET_DATA +# to an archival location. $MAGNET_DATA is passed as first param. +# The script is guarented to be called when MAGNeT is NOT running. +# This var may be set to empty +MAGNET_ARCHIVE="$MAGNET_BIN/magnet.copy" + +# End of configurable options +################################################################## + +################################################################## +# Set up some constants for this incarnation of MAGNeT + +# Name of MAGNeT /proc file (shouldn't ever change) +MAGNAME="/proc/net/magnet" + +# Pattern to match in /proc file for MAGNeT Device Name +MAGDEVNAME="Device Name" + +# Pattern to match in /proc file for MAGNeT Device Numbers +MAGDEVNUM="Device Number" + +# Names of MAGNeT executable files +MKMAGNET="mkmagnet" +MAGNETREAD="magnet-read" + +# End of constant setup +################################################################## + +# Set up "-n" functionality +if [ "$1" == "-n" ]; then + CMD="echo" +fi + +# First, determine if we are running a MAGNeTized kernel +ls $MAGNAME > /dev/null 2>&1 +if [ $? -ne 0 ]; then + echo "$0: Not running MAGNeTized kernel; aborting." + exit 1 +fi + +# Make sure we have proper MAGNeT device file (device name/number match) +# +DEVICE=`grep "$MAGDEVNAME" $MAGNAME | cut -f2 -d:` +DNUM=`grep "$MAGDEVNUM" $MAGNAME | cut -f2 -d:` +EXIST=`ls -l $DEVICE` > /dev/null 2>&1 +NEWDEV=$? +DEVNUM=`echo $EXIST | cut -f5-6 -d" "` +if [ $NEWDEV -ne 0 -o " $DEVNUM" != "$DNUM" ]; then + $CMD rm -f $DEVICE + $CMD mknod $DEVICE c $DMAJOR `echo $DNUM | awk 'BEGIN {FS=","}{print $1 $2}'` +fi + +# Generate a unique file name for the storage file +HOST=`hostname` +UNIQUE=`date +"%d%b%Y-%H%M%S"` +FNAME="$MAGNET_DATA/$HOST.$UNIQUE" + +# Make the new storage file +$CMD $MAGNET_BIN/$MKMAGNET $FNAME `echo "1024 1024 $MAGNET_SAVESIZE * * p" | dc` + +# There's a chance (ie, we're running from cron) that magnet-read is +# currently running. If so, send the TERM signal to it (this +# works for multiple running instances) +MPID=`ps -e | awk '{ if ($4 == "'$MAGNETREAD'") {print $1} }'` +if [ "$MPID" != "" ]; then + $CMD kill $MPID +fi + +# Compress and Archive latest MAGNeT storage file(s) +for STORE in `ls $MAGNET_DATA | grep -v ".gz" | grep -v $HOST.$UNIQUE` +do + $CMD gzip $MAGNET_DATA/$STORE +done +# Call Archive script to off-load the old storage files +if [ "$MAGNET_ARCHIVE" != "" ]; then + $CMD $MAGNET_ARCHIVE $MAGNET_DATA +fi + +# All's ready; re-run MAGNeT +$CMD $MAGNET_BIN/$MAGNETREAD $MAGNET_OPTS $DEVICE $FNAME & diff -PBbruw linux/net/magnet/mkmagnet.c magnet/net/magnet/mkmagnet.c --- linux/net/magnet/mkmagnet.c Wed Dec 31 17:00:00 1969 +++ magnet/net/magnet/mkmagnet.c Wed Sep 26 13:08:03 2001 @@ -0,0 +1,68 @@ +/* + MAGNeT - Monitor for Application-Generated Network Traffic + - MAGNeT storage file creator + - Written by Jeff Hay, radiant-software@lanl.gov +*/ + +/* This is an application, distributed with the kernel source. Therefore, its + likely to be built with the default kernel Makefiles, which define the + __KERNEL__ flag. Since we don't really want that, undefine it for us. */ +#ifdef __KERNEL__ +#undef __KERNEL__ +#endif + +#include +#include +#include + +#define BufferSize 1024*1024 + +int main(int argc, char *argv[]) +{ + FILE *OutFile; + int FinalSize = 0, Steps = 0; + void *DataSpace; + + printf("%s: MAGNeT Data Storage File Maker\n\n", argv[0]); + + if (argc < 3) { + fprintf(stderr," syntax: %s \n", + argv[0]); + return -1; + } + + if ((FinalSize = atoi(argv[2])) <= 0) { + fprintf(stderr, "%s: \"%s\" is not a valid file byte count; aborting.\n", + argv[0], argv[1]); + return -2; + } + + if (!(OutFile = fopen(argv[1], "wb"))) { + fprintf(stderr, "%s: Can not open \"%s\" for writing/creating\n", + argv[0], argv[1]); + return -1; + } + + if ((DataSpace = malloc(BufferSize)) == NULL) { + fprintf(stderr, "%s: Out of Memory; Can not create data store!\n", + argv[0]); + return -3; + } + printf("\tGenerating space for %d bytes of data\n", FinalSize); + + memset(DataSpace, 0x0FE, BufferSize); + Steps = 0; + while (FinalSize > (BufferSize)) { + Steps++; + FinalSize -= BufferSize; + } + + while (Steps-- > 0) { + fwrite(DataSpace, BufferSize, 1, OutFile); + } + fwrite(DataSpace, FinalSize, 1, OutFile); + + fclose(OutFile); + free(DataSpace); + return 0; +} /* main() */ diff -PBbruw linux/net/socket.c magnet/net/socket.c --- linux/net/socket.c Wed Apr 25 17:13:50 2001 +++ magnet/net/socket.c Wed Sep 26 13:08:03 2001 @@ -87,6 +87,10 @@ #include #include +#ifdef CONFIG_MAGNET +#include +#endif + static int sock_no_open(struct inode *irrelevant, struct file *dontcare); static loff_t sock_lseek(struct file *file, loff_t offset, int whence); static ssize_t sock_read(struct file *file, char *buf, @@ -509,7 +513,15 @@ err = scm_send(sock, msg, &scm); if (err >= 0) { +#ifdef CONFIG_MAGNET + magnet_add(sock->sk, MAGNET_SOCK_SENDCALL, size, + MAGNET_NO_DATA); +#endif err = sock->ops->sendmsg(sock, msg, size, &scm); +#ifdef CONFIG_MAGNET + magnet_add(sock->sk, MAGNET_SOCK_SEND, size, + MAGNET_NO_DATA); +#endif scm_destroy(&scm); } return err; @@ -518,10 +530,15 @@ int sock_recvmsg(struct socket *sock, struct msghdr *msg, int size, int flags) { struct scm_cookie scm; - memset(&scm, 0, sizeof(scm)); +#ifdef CONFIG_MAGNET + magnet_add(sock->sk, MAGNET_SOCK_RECVCALL, size, MAGNET_NO_DATA); +#endif size = sock->ops->recvmsg(sock, msg, size, flags, &scm); +#ifdef CONFIG_MAGNET + magnet_add(sock->sk, MAGNET_SOCK_RECV, size, MAGNET_NO_DATA); +#endif if (size >= 0) scm_recv(sock, msg, &scm, flags); @@ -1701,6 +1718,14 @@ #ifdef CONFIG_WAN_ROUTER wanrouter_init(); +#endif + + /* + * Socket Instrumentation Interface. + */ + +#ifdef CONFIG_MAGNET + magnet_init(); #endif /*