Re: Questions on Polipo capabilities and potential capabilities

Wesley,

It's interesting stuff that you're trying to do.

First of all, there is one thing that is not negociable: I will not
implement censoring of Javascript in Polipo.  Polipo only works at the
HTTP level -- it never parses entity bodies.  It would be easy to have
Polipo censor all instances with a MIME type of ``application/javascript'',
or all URLs that end in ``.js''.  Unfortunately, most Javascript
nowadays is embedded within HTML, and censoring it requires parsing
the HTML to find out the bits that are Javascript.  That is something
I'm definitely not going to implement in Polipo.  Sorry for that.

If you're still willing to consider Polipo with the above caveat, then
all of your other feature requests are quite reasonable.  Right now,
I'm in feature freeze, as I want to release 1.0.0, which I feel is
long overdue (there's one bug left that I've fixed today but not yet
pushed to the public repo, and one extra feature I want to put in).
After that, I'm quite willing to work on your feature list.

> (1) The problem with using cron to kill the polipo instance on the Linux
> server is that it is too crude.  I need a more graceful way of telling the
> user that the proxy is no longer functioning.  Ideally the user should be
> taken to a URL in this case, perhaps one defined in the config file or on
> the startup command line.

Actually, the disableProxy variable does exactly that; all you need is
the ability to customise the error message given when the proxy is
disabled (so it can redirect to the internal web server, or
a different web server).  As David suggested, you can then reconfigure
Polipo from cron.

Have a look at client.c:936; you'll want to replace it with something like

  httpClientNoticeErrorHeaders(request, 302,
                               internAtom("Proxying disabled"),
                               internAtomF("\r\nLocation: %s",
                                           "http://whatever";));

(If you want me to accept your patch in Polipo, this will need to be
configurable; have a look at the function preinitHttp and the
CONFIG_VARIABLE_SETTABLE macro.)

The only issue left is how to reconfigure Polipo at runtime, which can
either be done by rewriting the config file, or by POST-ing to the
configuration interface.  Unfortunately, Polipo doesn't currently have
proper authentication procedures for the configuration interface
(authorisation to reconfigure is limited to the disableConfiguration
variable).
  
> (2) The more I think about this, the more I believe we do need a
> leaky-bucket algorithm, which would effectively function similar to the
> "kill at time x" but in place of it, as a sort of "kill at traffic volume
> x".  Again, could you give me a pointer on where we would have to do the
> measuring of the bytes of traffic being served?

You're not specifying what you want exactly -- client or server-side
limiting?  In other words, do you want to limit the amount of data you
pull from remote servers, or the amount of data you push to clients?

As Polipo does caching, the two can be very different.

> (3) We cannot rely on the user to disable Javascript.

Sorry.

> (4) We need this too - and a way to manipulate others headers besides
> User-Agent :)

No problem, it's trivial.

> (5) Stunnel might work - we're going to put it through some tests.

As David mentioned, it'll probably be simpler for your users than
using a full-fledged VPN -- they won't need to install any new
software, they'll just set their HTTP proxy to https://whatever.

Alternatively, I could think about making Polipo grok SSL on the
client side natively, but I'm not sure I want to open that particular
bag of worms.

> As for http://www.privacy-ecosystem.com/ we are self-funded,

That's not quite clear to me.  Who's ``we''?

Paradoxically, in order to provide a trustworthy privacy infrastructure
you need to give up your own privacy and fully disclose to your users
where the money is coming from.  Are you an individual spending your
own salary on providing the infrastructure?  Are you a legally
recognised non-profit?  Are you a for-profit company?  If so, who are
your shareholders?

> Any chance of paying you, or someone, to do any of the above enhancements
> if all else looks good once we have finished our testing?

I will be glad to accept money (either for me or, if you're wealthy
enough, funding for a student) for the work described above.  If it's
okay with you, I'd rather discuss this kind of issues off-list.

                                        Juliusz

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV