osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

basic question on how "import" works


On 19Jan2019 11:42, joseph pareti <joepareti54 at gmail.com> wrote:
>[*u23885 at c009 3_NeuralNetworks]$ cat foo.py*
>from __future__ import print_function
>
># Import MNIST data
>from tensorflow.examples.tutorials.mnist import input_data
>mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)
>
>[u23885 at c009 3_NeuralNetworks]$
>when the above code is executed, equivalent of
>*$ python foo.py*
>
>the output is as follows:
>...
>Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
>Extracting /tmp/data/train-images-idx3-ubyte.gz
>Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
>Extracting /tmp/data/train-labels-idx1-ubyte.gz
>Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
>Extracting /tmp/data/t10k-images-idx3-ubyte.gz
>Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
>Extracting /tmp/data/t10k-labels-idx1-ubyte.gz
>...
>My questions are:
>
>   1. where are the write / print statements that display *Successfully
>   downloaded, extracting ...*

I would expect these to be in the tensorflow package. Find out where 
that is installed, then grep the source for "Successfully".

It is not uncommon for programmes to set their logging threshlds to emit 
"INFO" level logs if the output is a terminal, and possibly only 
"WARNING" or higher otherwise, so as to provide progress reporting to a 
person but less noise in a batch environment. So you may just be seeing 
info messages reporting progress. And it is likely that tere will be 
some mechanism to tune how noisy that it.

>   2. I don't see any such files in /tmp/data on my system, why?

These are probably temporary scratch files. The module will be fetching 
data files into the "/tmp/data" directory (which you have specified) and 
unpacking and importing their data. Once that is done the file is 
presumably discarded.

If you cd into that directory in another terminal and run "ls" commands 
regularly _during_ the import phase, you should see the filesappear and 
disappear as they are fetched, processed and discarded.

Cheers,
Cameron Simpson <cs at cskk.id.au>