There are many comparisons of PHP and J2EE performance out there. Many of the comparisons are either based on theoretical guesses or do some kind of CPU / Memory usage benchmarking while running various algorithms. It just doesn’t make sense. Most web pages are not running algorithms, they take requests, access a database enforce business rules and output html content. I decided to take a more realistic approach: build a data driven web application and load it until it breaks!
There is very much emphasis these days on decreasing development costs, so in order to maintain the realistic approach I used two frameworks for speedy development namely: CakePHP and JPA.
CakePHP has been developed to answer this need. It’s mission is to make developing a web application very simple. In fact, with a little of detail about the database schema, CakePHP can “Bake” a simple CRUD (Create, Read, Update and Delete) in minutes. This framework is very easy to use and very popular so I selected it to build the PHP portion of the test. This is a practical approach since it is likely that a developer would use a framework such as this to speed up development.
Java has also responded with JPA (Java Persistence Architecture). Using NetBeans, one can build the same simple CRUD application in minutes. It is amazingly easy to use and the outputted web application is fully functional. Again, this is a practical approach as a development team may use this as a starting point for the project.
DISCLAIMER:
This is not a scientific study of performance of either system! It is simply an experiment to see baseline performance expectations. I have done absolutely NO OPTIMIZATION to either environment. I understand I can increase the performance of either J2EE or PHP by tweaking this or that, but this test is not to see how fast I can get it. It is to see what is the limitation “out of the box”. Since many web apps may not be optimized completely, I believe this is a realistic test.
The testing software:
I coded a web load tester in Java to gather statistics on each framework’s performance. My point was not to measure memory usage or CPU cycles; rather, test user experience in time units. What result does the user see? How long does it take for the page to load after a request? And Does the request result in an error? Given n hardware and OS, what is the performance of either system? To do this, I simply made a Java program to spawn off threads each requesting a page with a list of records followed by a page retrieving a specific record. The same database was used and same MySQL server was used. Basically, everything is the same, the tests were even run on the same day and no reboot was performed.
PHP:
I ran the PHP test several times and took the best result.
Threads | Min Time | Max Time | Total Time | Average Time | Errors |
1 | 125 | 265 | 3057 | 152 | 0 |
6 | 179 | 2345 | 46952 | 390.8333333 | 0 |
11 | 283 | 1109 | 138164 | 627.6363636 | 0 |
16 | 437 | 12846 | 517952 | 1618.0625 | 0 |
21 | 496 | 13291 | 766749 | 1825.047619 | 0 |
26 | 583 | 254358 | 8193927 | 15757.07692 | 15 |
31 | 700 | 3067 | 1149899 | 1854.258065 | 0 |
36 | 775 | 32630 | 4874280 | 6769.333333 | 36 |
41 | 1000 | 30827 | 3221159 | 3927.780488 | 0 |
46 | 1183 | 91189 | 11254841 | 12233.06522 | 46 |
All Times are expressed in milliseconds.
Observations: CPU maxed out very quickly and the responsiveness became a problem as the server got loaded down. Many requests produced errors instead of actual content. There was very little consistency between one test and another.
I thought my client tester Java app may be loading down the CPU, so I moved it to a different machine on the same LAN to give more resources to Apache and PHP. Here is the result.
Threads | Min Time | Max Time | Total Time | Average Time | Errors |
1 | 140 | 297 | 3339 | 166 | 0 |
6 | 226 | 15493 | 140379 | 1169.333333 | 0 |
11 | 338 | 22823 | 537552 | 2443 | 0 |
16 | 435 | 23107 | 995587 | 3110.625 | 0 |
21 | 503 | 19681 | 1415508 | 3369.809524 | 10 |
26 | 668 | 18034 | 1249250 | 2401.961538 | 0 |
31 | 733 | 3441 | 1239943 | 1999.451613 | 0 |
36 | 834 | 45972 | 7613776 | 10574.22222 | 60 |
41 | 1010 | 41085 | 7637793 | 9313.878049 | 57 |
46 | 1031 | 4881 | 2653889 | 2884.195652 | 0 |
This is being performed in windows 7, you can see the complexity of the operating system’s control of resources. Since many people say that PHP works best on Linux, I decided to test it on my CentOS server. Although, the server has significantly less CPU power and memory, it is a really bare bones system with almost nothing running except Apache and MySql. The server does not even have X windows installed.
Threads | Min Time | Max Time | Total Time | Average Time | Errors |
1 | 253 | 1699 | 7050 | 352 | 0 |
6 | 1270 | 1651 | 185867 | 1548.333333 | 0 |
11 | 2441 | 2996 | 618783 | 2812.181818 | 0 |
16 | 3539 | 5827 | 1310252 | 4094.125 | 0 |
21 | 4797 | 6500 | 2294898 | 5463.47619 | 0 |
26 | 5435 | 9795 | 3437732 | 6610.461538 | 0 |
31 | 836 | 107276 | 7641826 | 12325.06452 | 0 |
BRICK WALL |
After 31 threads, the system hit a brick wall where there were so many httpd processes swapping memory, the system stopped responding. It is interesting to see how linear Linux’s performance is compared to windows.
J2EE
I did the same as the first PHP test with J2ee. Again, I ran it several times and picked the best one:
Threads | Min Time | Max Time | Total Time | Average Time | Errors |
1 | 15 | 94 | 640 | 32 | 0 |
6 | 15 | 241 | 7052 | 58.5 | 0 |
11 | 19 | 448 | 24094 | 109 | 0 |
16 | 28 | 334 | 39152 | 121.9375 | 0 |
21 | 28 | 359 | 46844 | 111 | 0 |
26 | 51 | 197 | 56446 | 108 | 0 |
31 | 42 | 513 | 102630 | 165.0645161 | 0 |
36 | 61 | 202 | 82723 | 114.4166667 | 0 |
41 | 76 | 247 | 116645 | 141.7804878 | 0 |
46 | 73 | 211 | 127305 | 137.8695652 | 0 |
Observations:
J2EE handled loading very well. I really couldn’t believe how well it handled the test and how little delay there was. J2EE was very consistent and no errors or brick wall was reached. I saw no reason to try to increase performance by moving the client application or trying Linux. The results were really good on the current configuration.
Although I attempted to increase PHP performance by offering several different execution environments, it could not perform nearly as well as J2EE. J2EE is a managed execution environment while PHP relies on the CGI model. PHP has great promise because of the huge development community; however, because it is not a managed environment, and relies on forking a new process for every request, it cannot perform under load like J2EE. Not to say PHP can’t be used in high traffic sites, in fact it is used in several high profile websites; however, it will take more work and hardware to make PHP handle loads like J2EE.
Below is an overlay of the best runs of PHP and J2EE.