HTTPS Everywhere uses too much memory

[bcroq]

Member note: One straightforward fix for this is described here (issue comment), along with an example PR in the Firefox codebase

I use HTTPS Everywhere Chrome 43.0.2357.65 (64-bit Linux) and the extension eats a little more than 100MB of memory. Isn't it too much for this kind of extension?

[reedy]

What do you mean by "this kind of extension"?

Noting the raw XML rulesets totalling 62M

[bcroq]

When I wrote "this kind of extension" I had uBlock in mind. This extension has 53869 filters on my computer and uses 68MB of memory.

[jsha]

@bcrow: Yes, my gut feeling is that HTTPS Everywhere can and should be optimized for better memory and CPU use. We haven't had the time to do it yet, but would very much welcome help from someone who is experienced at profiling extensions.

The uBlock number you quote is actually encouraging! HTTPS Everywhere has 14.5k 'rulesets', but each ruleset contains many targets and rules, which are presumably more akin to uBlock's filters. There are 39361 rules and 41287 targets, for a total of ~80k entities. The memory usage for those entities sounds proportional to uBlock, which is a surprise to me. Still, we can probably do a better job of keeping inactive rulesets out of memory.

[foresto]

For the record, this is not a Chrome-specific issue. It might be worth editing the title of this report accordingly.

In my Firefox profile, HTTPS Everywhere 5.0.5 is using almost seven times as much memory as uBlock Origin 0.9.8.2.

[jsantell]

Scoping this out for Firefox, will report back what I find

[imdjh]

Taking 100+ MiB memory upon opening only the 'New Tab'(which is taken ~70 MiB).
FYI: 44.0.2403.89 chromium on Debian 8.1, installed this via chrome webstore, this.version: 2015.7.17

[widefox]

Agree uBlock has shown the way forward to reduce memory usage. Can we get this prioritised?

[stokito]

Maybe it is possible to make online service API and make pre-request to fetch a rule?
Also it may be a small cache for last used domains.

[txtsd]

It would really help if this issue were prioritized. HTTPS Everywhere uses more RAM than any of my tabs or extensions on Chrome.

[jaens]

200M memory usage, this is getting a bit excessive.

[jaens]

I took a look at the heap snapshot, it's about 40MB, mostly going into the RuleSet objects. I think this could be reduced by:

• Not using RegExp objects unless necessary (as they use more memory than strings, especially if they are JIT-compiled).
• Not creating empty arrays in the rulesets, as these always consume heap (just set the collection to null when there are no entries).

There's probably value in reducing the peak memory usage as well by caching the parsed XML as JSON, so the heap wouldn't be oversized on startup.

[txtsd]

@jsha Is there any way you can elevate or prioritize this issue? I get that RAM isn't a problem for most people, but a lot of us still use old hardware with not that much RAM.

[Immortalin]

@txtsd I concur with your point. Doing any form of Clojure web development while using Chrome is close to impossible due to high memory consumption. And this a 4GB ram machine.

[blockoperation]

I was bored and had a go at doing some of these optimizations myself (blockoperation/https-everywhere@master...blockoperation:chromium-memory-usage).

This catches the really obvious cases (i.e. "^http:" for "from" in rules, cookie name patterns that match everything, etc), and almost halves the number of regexes on my setup (I tested these changes on the latest release in the web store, so the effect may be more or less with up to date rules) and reduces the heap snapshot by several MB.