The XRay HTTP Monitoring and Content Linearization Tool

Availability

XRay has been made available under the terms of the GNU Public License (GPL)

Installation

1. Obtain either the binary or source release of XRay 1.0 from the download page.
2. Configure your web browser to use XRay as a web proxy as explained below. The default port is 3002.
3. Run the jar file (tested under java 1.4):
   • java -jar <jar-file-name>

Configuration

The first time you run XRay 1.0, an empty xray.properties file will be written to your user directory and a small set of default application-wide properties will be loaded internally. These default settings will not be written to xray.properties. You should add an entry to xray.properties if and only if you wish to override a default setting corresponding to that entry. For example, if you would like to have XRay output linearized html rather than standard html, you would add the line:

parse_html=true

to xray.properties.

The complete set of properties together with their associated default values is documented here. Alternate values are obvious where supported.

Description

XRay 1.0 is a 100% pure Java, GUI enhanced, tool for monitoring and logging of all HTTP messages that transpire between any number of arbitrary web clients and any number of arbitrary web servers. XRay 1.0 runs much faster than XRay 0.5 and uses Log4J's LogFactor5 as a front end in place of the custom GUI included with XRay 0.5. Unlike the earlier release, XRay 1.0 does not perceptibly slow down your web browser. A future version of XRay will reinstitute the custom GUI, which contains some additional conveniences. Nevertheless, XRay 1.0 implements a number of features not supported by XRay 0.5: namely, the ability to decode chunked and/or gzipped content, and the option to transform an html or xml document containing nested elements into a plain-text, linear format that does not contain nested elements, while preserving the structure of the orginal document.

1. Configure any number of web browsers to use XRay as a proxy through some port on the machine on which XRay will run, say 3002. This is typically done via the main menu of your web browser. For example, if you are using Mozilla the sequence of mouse clicks is: Edit->Preferences->Advanced->Proxies. Then enable the "Manual Configuration" radio button and type "3002" (or whatever port you desire) into the appropriate text boxes. The process is essentially the same for all major web browsers.


2. Start XRay by running the jar file as shown above. If you have installed the latest version of Log4j, the GUI will appear (see the screenshot above), otherwise XRay will complain that it cannot find Log4J but will continue to run in text mode. You may capture all output to a text file by redirecting stdout to a file when starting XRay. Generate some data by using the attached client(s) to browse the web.


3. You may stop XRay at any time by pressing CTRL-C. Connected clients will no longer have access to the internet.
   

Currently Implemented Features

1. Full capture of all human-readable HTTP message content, including the headers and the body.


2. Proper decoding and translation of gzipped and/or chunked message bodies.


3. Suppression of non-human-readable binary content. Log records will not contain "garbage" characters.


4. The option to transform the content of an html or xml document into a linear format suitable for content filtering applications.


5. Because XRay is 100% java, it is platform independent.


6. XRay need not run on the same machine as the web client(s) it tracks.

Victims

1. The GUI has been appropriated from Log4J's LogFactor5 subproject.
2. The threading model was stolen from ProProxy.
3. The gzipped/chunked decoders include code ripped from SimpleWeb.
4. The content linearization code is from the book "Web Content Mining with Java" by Tony Loton.

Other Projects

• Bash Navigator

Author: Ben Tompkins
brtompkins@comcast.net