logo       


Re: wget and cookies: msg#00071

Subject: Re: wget and cookies
[ Please keep the bug address in the Cc, unless you explicitly wish to
  exclude it. ]

Grant Giddens <lggiddens@xxxxxxxxx> writes:

> The first thing I tried was:
> wget -vr -o bestbuy.log http://www.bestbuy.com
>
> That grabbed a few files, but the main index page basically said
> that the site couldn't be browsed because cookies were disabled.

I think that, for that to work, you need to log in to the site first.

> I went into IE and cleared all cookies and then visited te site.  I
> went to check on my cookies and I had 3 from best buy like:
[...]
> Of these 3 cookies, the first one looked the most important to me,
> so I copied it into a my wget directory and did:
>
> wget -vr -o bestbuy.log
> --load-cookies=grant@xxxxxxxxxxx[1].txt
> --save-cookies=grant@xxxxxxxxxxx[1].txt
> http://www.bestbuy.com

Why not just give all the cookies to Wget?  Although one can look more
important out of the bunch to you and me, you never know what the site
authors meant when they were setting the other two cookies.  Also, are
you sure that those files are in the correct cookie file format?

Another thing you might want to look out for are session cookies.
When you exit IE completely (with "ctrl-q" or something that closes
*all* browser windows), do you have to log in again to the bestbuy
site?  If that is the case, it means that the site uses "session
cookies" which don't get saved to the cookie file and which Wget
cannot pick up.

There are ways around session cookies, but let's check if they are in
fact the problem before turning the attention to them.

> I'm not sure what to try now.

Try providing a copy of the entire IE cookie jar to Wget.  (The manual
explains how to do that.)  If that doesn't work, check if the session
cookies are creating the problem.  If that is the problem, we'll fix
it.  Finally, if that is not the case either, you'll need to check
Wget's debug output to see if the correct cookies are being sent in
the first place.




Ruby Jobs
Java Jobs
Jobs in California
more...
what
job title, keywords
where
city, state, zip
jobs by job search
Search:
Java, servers, webhosting, windows, cisco ...
more...
<Prev in Thread] Current Thread [Next in Thread>
Google Custom Search

Recently Viewed:
encryption.gpg....    ietf.rfc822/199...    freebsd.devel.i...    lang.haskell.li...    mail.squirrelma...    web.zope.plone....    yellowdog.gener...    text.xml.xalan....    recreation.phot...    kde.devel.educa...    hardware.bus.ca...    printing.ghosts...    voip.peering/20...    assembly/2006-0...    org.user-groups...    culture.interne...    network.i2p/200...    boot-loaders.ya...    xfree86.render/...    qnx.openqnx.dev...    jakarta.velocit...    user-groups.pal...   
Home | blog view | USPTO Patent Archive | advertise | OSDir is an inevitable website. super tiny logo

Free Magazines

Cisco News
Receive a free quarterly e-newsletter with exclusive articles on how Cisco IT uses its own products and solutions to enable the business.
subscribe

Systems Management News, the newspaper for IT systems administration and data center managers! Each issue of Systems Management News is chock-full of news and analysis to help you understand what's happening in your field.
subscribe

The Enterprise Newsweekly eWeek is the essential technology information source for builders of e-business.
subscribe

Oracle Magazine Oracle Magazine contains technology strategy articles, sample code, tips, Oracle and partner news, how to articles for developers and DBAs, and more. Oracle (NASDAQ: ORCL) is the world's largest enterprise software company.
subscribe

Total Telecom Total Telecom is "The Economist of the communications industry".
subscribe