Feature #164
Optimization work
0%
Description
Working on optimizing philo - see also: http://www.codinghorror.com/blog/2011/06/performance-is-a-feature.html
Will use this issue to track progress.
History
Updated by Stephen Burrows almost 13 years ago
- Category set to Core
Commit 5da36a2 test times in seconds (max, min, avg):
Before: 11.373934 05.021761 05.505069 After: 07.170535 03.784184 04.082582
Updated by Stephen Burrows almost 13 years ago
Incidentally, tests are run in python with urllib. The test page is accessed 100 times and is (probably) not representative of every feature that philo offers.
Updated by Stephen Burrows almost 13 years ago
Commit 6164c1b test times in seconds (max, min, avg), reusing the 'after' statistics from the previous commit.
Before: 07.170535 03.784184 04.082582 After: 05.391495 03.684451 03.785000
Updated by Stephen Burrows almost 13 years ago
Commit 7ebd19f test times in seconds (max, min, avg), reusing the 'after' statistics from the previous commit.
Before: 07.170535 03.784184 04.082582 After: 07.660383 03.617115 03.862377
No significant overall difference, though the first page load (when caches are warmed) seems slightly faster.
Updated by Stephen Burrows almost 13 years ago
That last one should've read:
Before: 05.391495 03.684451 03.785000 After: 07.660383 03.617115 03.862377
Updated by Stephen Burrows almost 13 years ago
[optimization adbb2ab] Added memoization (optional but enabled by default) to TreeEntity.get_path.
This optimization mostly helps pages that make heavy use of shipherd, node_url, and (more generally) node.get_absolute_url. The principal is that in the course of a Node (or other TreeEntity subclass) instance's life, its path is not likely to change. In cases where change is important, developers can pass in memoize=False to force evaluation.
develop: 11.373934 05.021761 05.505069
5da36a2: 07.170535 03.784184 04.082582
6164c1b: 05.391495 03.684451 03.785000
7ebd19f: 07.660383 03.617115 03.862377
adbb2ab: 04.043702 03.158195 03.265863
Updated by Stephen Burrows almost 13 years ago
Cacheless shipherd (fa49fa4) vs. optimization otherwise (c679970):
c679970: 06.403735 03.060383 03.146596 fa49fa4: 04.328504 02.848535 02.933984
Updated by Stephen Burrows almost 13 years ago
Unfortunately, those results were bogus; the cacheless shipherd code was just not running, because of template filter errors. Once those were corrected, cacheless lost by a wide margin. Currently trying to build a better cache.
Updated by Stephen Burrows almost 13 years ago
Optimization 13aa220 - this particular commit made single-level navigation structures (as are used on the test site) much faster.
a05c936 - this is a version that actually works. ;-)
Better cache
13aa220: 04.884354 02.901736 02.995597 a05c936: 04.300317 02.871088 02.943538
Updated by Stephen Burrows almost 13 years ago
Turns out that a05c936 actually still had some slight issues, but they did not affect performance. In cases where a navigation item didn't have a target node, the navigation was evaluated, but not displayed. Correcting the issue actually seems to have improved performance slightly, though it could be slight a glitch in the results.
7b3b752: 04.403769 02.803217 02.883353
Updated by Stephen Burrows almost 13 years ago
I think that it might be possible to do an optimization for shipherd and sobol similar to the one described in slides 41-44 of the first presentation at http://justcramer.com/2011/06/24/europython/. We're already mostly there. I'm not sure, though, whether the technique really applies to the problem at hand.
Updated by Stephen Burrows almost 13 years ago
12c9cc7: 04.337620 02.821180 02.913605 - fast containers with reference prefetching 829ebbd: 04.480525 02.766281 02.836390 - fast containers without reference prefetching
It should be noted that reference prefetching would probably be more efficient given a template that referenced many instances of the same model; however, it has (in uses of philo I have seen) been more common to reference various models once each. In this case, prefetching is slower.
Updated by Stephen Burrows almost 13 years ago
eeb8a55: 04.446627 02.752377 02.837770 - now caches philo-root reversal in Node.construct_url
At this point it should be mentioned that these tests are being run using the locmem cache backend, which is much slower than, say, memcached, and which consequently results in comparable performance rather than improved performance for this particular commit.
Updated by Stephen Burrows over 12 years ago
b7bb8ec: 04.439187 02.615551 02.690186 183b0a2: 04.157114 02.611704 02.675582
The difference in 183b0a2 (which introduced caching of target urls on TargetURLModel instances) is not significant enough to be considered a clear improvement; however, I think that may have something to do with the site that was being used to test this. I'm marking this topic as resolved for now. It's possible that there are some more optimizations that can be found, but they don't need to be found right now.
Updated by Stephen Burrows over 12 years ago
- Status changed from In Progress to Closed
Marking as closed for now.