= Setup instructions =

*Windows Version: There is a local copy here *Linux Version should came bundled with your Linux distribution.

:refresh.py 12 Kb 20/01/2007 00:58 -a--
:refreshdb.py 4 Kb 19/01/2007 21:55 -a--</code>
:<code>util.py 3 Kb 19/01/2007 21:51 -a--</code>
:<code>alive.py 3 Kb 18/01/2007 15:05 -a--</code>
:<code>**client.ini 0 Kb 04/02/2007 14:37 -a--**</code>
:<code>**client.bat 0 Kb 04/02/2007 14:41 -a--**</code>
:<code>**CK.RTN 0 Kb 31/01/2007 13:31 -a--**</code>

[[http://www.iberonesia.net/archives/campaigns/client.zip|You can download here client.zip]]

===== Edit the configuration file is the client.ini  =====
:[[global]]
::brewerid=037
::working_dir=.
::input=c:\brewer\data
::servers=iberonesia
::proxy=proxy.xxx.xxx:port
:[[iberonesia]]
::url=http://www.iberonesia.net/brewer
::db=www.iberonesia.net.db
::noproxy=yes

<br> You only have to edit the brewerid to your brewer. If you dont need a proxy comment the lines (put # at the beginnign of the line)

===== Edit client.bat program (calling by the routine Ck.rtn)  =====
client.bat

       python refresh.py -i **c:\brewer\data**

You only have to edit the brewer **(-b)** and the brewer data directory **(-i ).** <br>

Parameters -b and -i are not necessary if specified in client.ini file (recommended)

<br> if you want to save the errors and log edit the client.bat and put

       python refresh.py 1&gt;client.log 2&gt;client_error.log

A list of all the files in the directory of brewer is obtained.

===== Testing  =====
Open a comand window, change dir to the brewer program directory and execute client.bat. The first time it takes a while before you see the message..... from the brewer menu call the routine "ck"

   Important: restart the windows after the installation.

<br>

= Operating instructions =

===== How it works  =====
When all parameters are processed, it is known the files to send and the servers to send to. The program loops though the servers and tries the send all the files specified. The client maintains a local database storing md5sum and size of the files that has been transferred to the server, used to determine if a file needs to be sent.

==== Sending files to a server  ====
The protocol to send the files is:

:Check if the refresh database for the server exists. If not exists, the database is [[#Regenerating_local_database|(re)generated]].
:For each file:
::Check if the file needs to be sent. The client knows if a file needs to be sent comparing the (md5sum, size) stored in the local database with the (md5sum, size) of the actual file.
:::If the file entry in the local database does not exist, the file is sent
:::If any of the md5sum, size stored in the database differs from the ones in the actual file, the file is sent.
:::In other case (md5sum and sizes are equal), the file is not sent.

The transfers cannot be done file to file (would be too many) nor all the files simultaneously (too many data to transfer could cause a time out). So, the files are transferred in blocks. If a threshold size of files is reached, the files are packed in a zip file and sent to the server. Once the file is actually sent, the (md5sum, size) in the local database is updated.

==== Regenerating local database  ====
The protocol to regenerate the database is:

:The client sends to the server all the files that is willing to transfer (i.e., the files specified with the input parameter)
:For each file in the previous list, the server returns the appropiate (md5sum,size) tuple of the file stored in the server. If the server knows nothing about that file (i.e., it has never been transferred), no entry is returned.
:Each entry returned to the client is written to the local database.

This way, the local database stores the same info about the file that is stored in the server, so there is no need to query that info to the server for each file in the input, thus saving a lot of network time.

It is possible that the local database gets corrupted or out of sync with the server (e.g., a server is replaced and some files had been sent to the old server that are not in the new server). In that case, the database needs to be deleted, and so regenerated in the next execution.<br>

= Appendix =

===== Client configuration file  =====
The client configuration file is used by both the refresh.py and alive.py programs. The default configuration file is client.ini.

The file is a plain ASCII INI file with the [[http://en.wikipedia.org/wiki/INI_file|INI file syntax]]. The file consists of a global section and several servers sections, one for each server to send the files to.

The global section has the following properties:

*brewerid&nbsp;: brewer identifier of the brewer owner of the files to send.
*input&nbsp;: directory that contains the files to send.
*servers&nbsp;: comma separated list of server aliases. The files are sent to each server in this list.
*working_dir&nbsp;: directory where the local refresh database is stored.
*proxy&nbsp;: proxy url, if required. If not required, this property must not be present.

A server section is identified by the server alias (i.e. the alias is the name between the brackets), and it has the following properties:

*url&nbsp;: server url.
*db&nbsp;: local database filename. A local database (a simple data file) is created the first time client is run. The database path is determined by the rundir property and this db property.
*noproxy&nbsp;: if a proxy is specified in the global section, noproxy may be set to yes to use a direct connection (avoiding the proxy) to access this server.

Some of these properties may be overridden using command line parameters when executing refresh.py or alive.py.

Example:

<code></code>

<code></code>

<code></code>

<code></code>

<code>
 [[global]]
 brewerid=033
 input=c:\data
 servers=iberonesia
 working_dir=c:\refresh
 proxy=http://proxy.ciai.inm.es:3128
 # iberonesia server
 [[iberonesia]]
 url=http://www.iberonesia.net/brewer
 db=www.iberonesia.net.db
 noproxy=yes

Most of the refresh.py options may be configured though command line arguments or client.ini configuration file.

Any option specified in command line overrides the option in client.ini.

All options are shown executing the client with parameter -h<br>

0@@

The mandatory options are:

*working_dir&nbsp;: This option must be specified in client.ini. Directory where the local refresh database is stored. *input&nbsp;: File to process. If the given file is a directory, all files within will be processed. *brewerid&nbsp;: Brewer identifier of the brewer owner of the files to send. *servers&nbsp;: Comma separated list of server aliases. The files are sent to each server in this list.<br>

The easiest way to configure the client is to specify all options in client.ini file (like the one above) and execute refresh.py without arguments; and execute with arguments overriding the client.ini options if you want any unusual operation, such as sending only certain files. E.g. (using the client.ini above):<br>

1@@

To debug any problem, it is recommended to execute the client with a command line and using the -v parameter (the more v, the more debugging information shown, with -vvv the maximum).<br>