Network Performance Effects of HTTP/1.1, CSS1, and PNG

February 10, 1997

This talk obsoletes Talk of January 15

Henrik Frystyk Nielsen, W3C

Jim Gettys, W3C / Digital

Anselm Baird-Smith, W3C

Eric Prudhommeaux, W3C

Håkon Wium Lie, W3C

Chris Lilley, W3C,

http://www.w3.org/pub/WWW/Talks/970210HTTP/

Purpose of this Evaluation

Evaluate the gains of using HTTP/1.1 for basic web transport
Does HTTP/1.1 fulfill its design goals?

For the network? (network traffic/behavior)
For the end user? (perceived speed)

Test Persistent Connections, Pipelining
Tested two common situations

First Time Retrieval (Filling the cache)
Subsequent Retrievals (Cache validation)

Some investigation of HTTP/1.1 Content-Encoding (e.g. gzip compression)
Preliminary investigation of CSS1 and PNG's effects on network traffic
No investigation of HTTP/1.1 caching

Test Setup

Applications used for Testing

libwww 5.1 robot as client (C based)
Jigsaw 1.05 as server 1 (Java based)
Apache 1.2 beta 1 and 1.2b7 + patches as server 2 (C based)
Limited use of Netscape Communicator beta 1

Platforms used

Servers running on Sun Solaris (Sparc Ultra-1, Solaris 2.5)
Clients running on Digital Alphas (Digital UNIX 4.0, 4.0a),
PPP client data from Pentium Pro, NT Server 4.0

Test Setup cont.

HTTP Client Implementations

HTTP/1.0 with 6 simultaneous connections
HTTP/1.1 with 1 persistent connection
HTTP/1.1 with 1 persistent connection using pipelined requests
Netscape Communicator used in a single test case, to validate results

Server HTTP Implementations

HTTP/1.1 in all cases

Pipelining and Output Buffering

Pipelining allows multiple outstanding requests, reducing round trips

Responses are still serialized - difference is timing

Buffering packs TCPs segments better

Reduces number of packets (and server context switches) required for same "work"
Saves packets, CPU time, and ultimately elapsed time

When to Flush?

If the data in the output buffer exceeds 1K
If the data is buffered longer than N ms
If the application explicitly requests it

Experimenting with Nagle's algorithm

Turned on/off in both client and server
No differences seen in initial tests, but later tests showed its effect.

Recommend disabling Nagle (set TCP_NODELAY socket option)

Description of Tests

Mix of Microsoft and Netscape pages ("Microscape home page")

1 HTML document (~ 40K)
41 Inlined images ( ~ 130K)

Cache Load Test

1 GET request on HTML document
GET requests on 41 inlined images
Think:visit a site for the first time

Cache Validation Test

1 GET request on HTML document (for HTTP/1.0 test)
Simulated cache validation with 41 HEAD requests (for HTTP/1.0) test
Test against HTTP/1.1 used full persistent cache, ergo 42 GET requests using Conditional GET's and entity tags
Think: revisit a site you have been to before, or "reload everything" button on a browser

Network Environments Tested

High Bandwidth, Low Latency

LAN 10 Mbit Ethernet (RTT ~ 1ms)

High Bandwidth, High Latency

WAN between MIT and LBL (RTT ~ 75ms)

Low Bandwidth, High Latency

PPP over 28.8 modem (via telephone switch simulator, RTT ~ 150ms)

Results - LAN Load/Validation

Jigsaw - High Bandwidth, Low Latency
	Packets	bytes	time	eff.	Packets	bytes	time	eff.
	First time retrival				Cache validation
HTTP/1.0	455.2	187525.6	1.12	0.912	362.8	58993.0	0.76	0.803
HTTP/1.1	234.4	189938.0	1.32	0.953	88.4	16878.0	0.80	0.827
HTTP/1.1 Pipelined	168.0	189646.0	0.69	0.966	27.6	16878.0	0.52	0.939
HTTP/1.1 Pipelined and compression	140.4	158460.0	0.59	0.966	27.2	16873.0	0.47	0.939

Apache - High Bandwidth, Low Latency
	Packets	bytes	time	eff.	Packets	bytes	time	eff.
	First time retrival				Cache validation
HTTP/1.0	449.8	188237.4	1.11	0.913	339.4	59008.0	0.54	0.813
HTTP/1.1	232.8	187618.0	0.81	0.953	88.0	13731.0	0.39	0.796
HTTP/1.1 Pipelined	163.2	187618.0	0.52	0.966	24.4	13731.0	0.27	0.934

Results - WAN Load/Validation

Jigsaw - High Bandwidth, High Latency
	Packets	bytes	time	eff.	Packets	bytes	time	eff.
	First time retrival				Cache validation
HTTP/1.0	455.4	191808.2	4.02	0.913	339.6	60745.0	3.28	0.817
HTTP/1.1	254.4	190965.2	9.19	0.949	90.0	16916.4	5.34	0.825
HTTP/1.1 Pipelined	210.6	190635.8	3.22	0.958	26.8	17170.0	1.32	0.941
HTTP/1.1 Pipelined and compression	181.0	159032.4	3.18	0.956	27.8	16873.0	1.30	0.938

Apache - High Bandwidth, High Latency
	Packets	bytes	time [sec]	eff.	Packets	bytes	time	eff.
	First time retrival				Cache validation
HTTP/1.0	473.6	191385.4	5.49	0.910	340.6	59008.0	2.36	0.812
HTTP/1.1	252.0	188786.0	6.93	0.949	88.8	13755.2	4.73	0.795
HTTP/1.1 Pipelined	204.0	188811.2	2.91	0.959	25.2	13731.0	0.95	0.932

Results - PPP Load

Jigsaw - Low Bandwidth, High Latency
	Packets	bytes	time	eff.	Packets	bytes	time	eff.
	First time retrival				Cache validation
HTTP/1.0 **)	489	235027	65.05	-	-	-	-	-
HTTP/1.1	349.6	189458.0	63.82	0.931	129.0	16800.0	12.28	0.765
HTTP/1.1 Pipelined	286.0	190383.2	52.35	0.943-	32.0	16868.0	5.40	0.929

**) Netscape Communicator 4.0 beta 1 with max 4 simultaneous connections and HTTP/1.0 keep-alive connections. The Netscape HTTP client implementation uses the HTTP/1.0 Keep-Alive mechanism to allow for multiple HTTP messages to be transmitted on the same TCP connection.

Tools

Tools from elsewhere:

tcpdump
xplot
tcpshow

Tools we built:

Summary (in our tests)

Roughly half of the packets in HTTP/1.0 are TCP open/close

Such packets are not "congestion controlled"

Significant further gain by using buffering with pipelining

Validation may use as little as than 1/10 the packets of HTTP/1.0
Cache load may use less than 1/2 packets of HTTP/1.0

The mean size of a packet doubled in our tests
The mean number of packets in a TCP session increased between a factor of 2 and 10. This is less than one might naively expect, and is due to buffering that reduces the number of packets/session.
HTTP/1.1 is up to twice as fast as HTTP/1.0 (elapsed time)
Server performance also gains when using pipelining
Transport compression saved significant bandwidth
Recommend that Nagle is turned off

Summary (Continued)

Server performance also increases when using pipelining
Apache 1.2b7 + patches is now faster than Jigsaw
Total implementation time was about 2 people for two months
Client implementation required some care and effort; server implementation was very easy, though some care and output buffering makes a significant difference in performance

Summary (Style Sheets and PNG)

Biggest single bandwidth gains may be deployment of style sheets, by eliminating many small graphical elements

15 of 40 images in our sample were easy to convert to CSS
An additional four images are if the font is available
3 images if negative margins are allowed
3 images can be replaced by a mixture of CSS and images

Style sheets gain two ways:

The representation is smaller than image formats
and by reducing the number of protocol requests

PNG saved bandwidth

But not very much, in our tests (the images were mostly small)
For big images, PNG has significant size savings over GIF
there are other good reasons to adopt PNG - e.g. no patent problems and gamma correction

Future Work (worth doing)

We'll probably investigate:

Comparison of Deflate and modem compression
Investigation of binary/tokenized protocol representations

We'll probably not investigate:

Range requests could bear further investigation
Possible optimization of compression dictionaries
Quantification of CPU time savings
Time to Render
Better understanding of connection management policies
More recent trace data to help understand Web performance issues and connection management policies

Conclusions

Pipelined implementation required to outperform HTTP/1.0's elapsed time
Buffered pipelined implementation also greatly the reduced number of packets
In our tests, HTTP/1.1 w. pipelining using a single connection ALWAYS outperformed HTTP/1.0 using multiple connections
HTTP/1.1 will help the Internet's problems significantly, once deployed. The improvement in number of packets will be at least a factor of two, with much lower congestion
Users will see observably higher performance
HTTP/1.1 should be deployed as soon as possible