On 2005-03-21 15:32, gentoo@xxxxxxxxxxxxxxxxx wrote:
> *** This is not problem of wget, but your filesystem. Try to do
>
> touch
> search.ebay.de/wget_W0QQcatrefZ3DC6QQcoactionZ3DcompareQQcoentrypageZ3DsearchQQcopagenumZ3D1QQdfeZ3D20050024QQdfsZ3D20050024QQdfteZ3DQ2d1QQdftsZ3DQ2d1QQfltZ3D9QQfromZ3DR9QQfsooZ3D2QQfsopZ3D2QQsaetmZ3D1111396614QQsojsZ3D1QQsspagenameZ3DADMEQ3aBQ3aSSQ3aDEQ3a21QQversionZ3D2.html
I'm very sure that my file system has some limits somewhere - but I
suppose a web server may create virtual URLs which will be too long or
will include illegal characters for almost any file system around.
The file name here might get repaired by some regex, e.g.
wget_?&catref=C6&coaction=compare&coentrypage=search&copagenum=1&dfte=Q2d1&dfts=Q2d1&flt=9&from=R9&fsoo=2&fsop=2&saetm=1111396614&sojs=1&sspagename=ADMEQ3aBQ3aSSQ3aDEQ3a21&version=2.html
However, I'd be comfortable enough with some fixed length or char
limitation, such as a 'trim' extension:
-tc, --trimcharacter char cut filename after character, such as "_"
-tl, --trimlength num cut filename after num characters
-ts, --trimsuffix num digits used for incremented cut filenames
-tt, --trimtable file log trimmed file name and original to file
For the moment I'd be happy enough with saving to a md5.html checksum as
filename instead of a filename too long for my fs.
The output log could tell me about the shrinked and the original
filename.
> > search.ebay.de and then download the links to cgi.ebay.de in one:
> >
> > wget -kxrE -l1 -D cgi.ebay.de -H $URL
>
> *** maybe to create SHA1 sum of the request and store the result in this file
> (but you will not know what was the original request, if you don't create some
> DB of requests). Or do just simple counting
>
> URL="....."
> sha1sum="$( echo -n "$URL" | sha1sum )"
> echo "$sha1sum $URL" >> SHA1-URL.db
> wget -O sha1sum.html [other options] "$URL"
>
> or
>
> URL="...."
> i=0
> echo "$i $URL" >> URL.db
> wget -O search-$i.html "$URL"
>
> Could be this your solution?
Nice idea - I'll give it a try. However, it does not answer the -D problem
itself. I'm afraid this does require some further awk/sed processing of
the result?
Thanks,
Martin
|
| 
