Attributor: Abuse other people’s resources with confidence

Posted 2008-03-12 in Spam by Johann.

Attributor constantly scans billions of web pages to find copies of your content across the Internet

…in stealth mode, of course.

$ egrep '^64.41.145.' <logfile>
64.41.145.193 … "GET /blog/?flavor=rss2 HTTP/1.1" 301 0 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 1.0.3705)" "-"
64.41.145.193 … "GET /blog/ HTTP/1.1" 200 18934 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 1.0.3705)" "-"

Their other crawler is slightly more “open”:

64.41.145.167 … "GET /software/stringsearch/ HTTP/1.0" 403 4252 "…" "Mozilla/5.0 (compatible; attributor/1.13.2 +http://www.attributor.com)" "-"

Block Attributor

Let’s perform the standard RDNS trick.

(Asked whois.arin.net:43 about +64.41.145.193)
…
 OrgName:    Attributor Corporation 
 OrgID:      ATTRI 
 Address:    1779 Woodside Road  Suite 200 
 Address:    ATTN Adrian McDermott 
 City:       REDWOOD CITY 
 StateProv:  CA 
 PostalCode: 94061 
 Country:    US 
 NetRange:   64.41.145.0 - 64.41.145.255 
 CIDR:       64.41.145.0/24 

Attributor’s netblock: 64.41.145.0/24.

Step-by-Step Web Application Performance Testing and Tuning

Posted 2008-03-06 in Programming by Johann.

1. Document current performance

• Test the entire system. I test the performance of blojsom (blog software) running in Orion (application server) proxied by lighttpd.
• Test only the parts that can be made faster. I use lighttpd for static files so I would never include static files when optimizing Servlet/JSP performance.
• Never touch libraries. Third-party or open-source libraries can be performance bottlenecks. Patching libraries – especially if you do not have the source code – can become a maintainance nightmare. Look for replacements and newer versions where possible.
• Test the real thing. Most of the time, this will be a dream though. My 2.4 GHz Xeon desktop machine does 0.7 requests/s while my HostEurope VPS manages over 2.5 requests/s. Hey, it’s just a virtual server so it must be slower, right? Wrong.
• Test with the production operating system. Certain operations are expensive under Windows and cheap somewhere else and vice versa.

I use Microsoft’s WAST to test performance. The Microsoft Web Application Stress Tool is ages old, but I like it for several reasons:

• Control over request headers.
• Creates obscene amounts of requests (several 1000/s).
• Free with a good GUI.

2. RAM is cheap

A lot of software packages offer in-memory caching. Use it! Keep in mind you might be changing factors like

• Speed of garbage creation. By caching, you can change the memory allocation behaviour of an application. Applications that dominantly generate new objects (per-request) will suddenly keep objects for much longer.
• Garbage collection speed. If you keep more objects in RAM, your memory usage will grow which will lead to more frequent garbage collections (if you do not increase the memory allocation) and longer running garbage collections (if you do). Enable garbage collection logging and use GCViewer to check how much time is spent in garbage collection. 95+ % is very good, 75 % is not good I would say.

Like everything else, caches have bugs. RAM file systems exist.

In my case, using caches increased the performance from 0.7 requests/s to 5.4 requests/s.

3. Profile

Profilers will tell you how much time is spent where. They come in all varieties but, in the end, they all do the same thing.

• Profile as many classes as possible. If 90 % of the CPU time is spent in system classes but you don’t profile these, you will optimize the application classes – where only 10 % is spent.
• Don’t profile classes you cannot change. Exclude the class hierarchy of your application server.

In this example, you can see that most of the CPU time is spent in the VelocityEngine#init method. Looking into the code, I discovered that a new VelocityEngine was generated for each request. Oops.

After caching the VelocityEngine, the Velocity page creation takes 10 % CPU time instead of 55 % and the performance is 7.7 requests/s instead of 5.3 requests/s. But – we have a new performance hot spot in the StandardFetcher class.

As you see, each performance optimization step shifts the bottleneck to a new place.

4. Stop

In an ideal world, web pages would come preinstalled on the visitor’s computer. In the end, nobody can tell the difference between a latency of 70 ms and 100 ms on the web.

Does it make sense to repeat these performance tuning steps? Yes, if you cannot improve performance by other means (and most of the time, you can) and if the performance is seriously bad (in that case, changing software might also be an option).

JavaScript Compression Alternatives to /packer/

Posted 2008-03-03 in JavaScript by Johann.

Squeezing the last bits out of your JavaScript files? Then you probably know /packer/. Maybe you even use it with YUI to automatically compress multiple JavaScripts into one file.

Recently, I discovered new alternatives to /packer/ and decided to compare them.

JavaScript compressors

The two new JavaScript crunchers I test are

• JSIntegration, a Java-based JavaScript packer with a Java Swing frontend. I used the JSA-20071021.jar file. My settings were: “Syntax compression” on, “text compression” on, “Compatible(IE5,NS3)” on, “auto confuse” off. The header comment was removed.
• JavaScript Utility V2 by Patrick J. O'Neil. I used the ISO-8859-1/UTF-8 setting.

Test files

The files I compressed were

1. The JavaScript for johannburkard.de. 73716 B uncompressed, 43170 B after YUI.
2. jQuery 1.2.3. 96763 B uncompressed, 53807 B after YUI.

I started with YUI compressed versions.

Results

johannburkard.de JavaScript

1. 23223 B (31 %) – JavaScript Utility
2. 25711 B (34 %) – JSA
3. 27988 B (38 %) – /packer/

jQuery 1.2.3

1. 25877 B (27 %) – JavaScript Utility
2. 28646 B (30 %) – JSA
3. 29606 B (31 %) – /packer/

Decompression times

While packed JavaScript files make your pages load faster, the browser still has to decompress the file. I didn’t benchmark all packers but the previous version of JavaScript Utility was about twice as slow as /packer/. After a recent update, JavaScript Utility is now about as fast as /packer/ (some informal performance numbers, OpenOffice document).

Edit: Neil Roberts suggested that I try ShrinkSafe instead of YUI. Here are the results:

johannburkard.de JavaScript

24689 B (33 %) – ShrinkSafe, then JavaScript Utility.

jQuery 1.2.3

27469 B (28 %) – ShrinkSafe, then JavaScript Utility.

3 comments

6 quick jQuery tips: Text manipulation, timers and elements

Posted 2008-02-26 in JavaScript by Johann.

This is the first of several jQuery snippet collections.

Remove a word with jQuery

The simple way – using regular expressions:

var el = $('#id');
el.html(el.html().replace(/word/ig, ""));

Test it!

jQuery timer callback functions

Want to call a method after a certain timeout?

window.setTimeout(function() {
 $('#id').empty();
}, 1000);

Remove this element one second after clicking it.

If you want to call a task periodically, use the Timer plugin for jQuery.

Verify that an element exists in jQuery

Simply test the .length property. Bonus information: This is used in inc.

if ($('#id').length) {
 // do stuff
}

Is there an element with an id of “top”?

jQuery not working in IE 5.0 or 5.5?

jQuery does not support older Internet Explorer versions. To make sure your users do not see JavaScript errors, edit your jquery.js file as follows:

// Put this before the original jQuery code
if (!(window.showModelessDialog && !document.implementation)) {
 (function() {

// Original jQuery code goes here

// Put this after the jQuery code
 })();
}

How to use a plugin with jQuery?

jQuery plugins are included on the page after the main jquery.js file:

<script type="text/javascript" src="jquery-1.1.4.js"></script>
<script type="text/javascript" src="jquery.roflcopter-1.0.js"></script>
<script type="text/javascript" src="jquery.lolcode-2.4.js"></script>

This is the beginner’s version. The advanced version is copying all your JavaScript files into a single file and then compressing it with YUI and /packer/.

$('<div>hello<\/div>').appendTo(document.body);

Pages

Page 7 · Page 8 · Page 9 · Page 10 · Page 11 · Page 12 · Page 13 · Next Page »