Text only mirror site

Today I have finished writing a very dirty shell script that automatically creates a text version of this site once a month. That is to say a mirror site updated on a monthly basis. You can take a look at it here

You may wonder why the text version is not a 'real text' version but a cheap copy of the actual site. Well there are several mandatory reasons for that:

1st The hosting server demands an index document in .htm format
2nd I have tried to process html files with html2text but it unfortunately does not render optimal results.

Well if anyone is interested in the code I'll publish it here. The performance is as follows:

- The script first tests working conditions and makes clean.
- Then it downloads the entire site and turns it into text. Both things using for loops.
- After that it uploads it to the server.
- Finally it sends me and e-mail to confirm success.

TODO: -Consider creating a section of useful scripts???
           - Test mirror site with w3m and lynx. (I have tested it with elinks and it works great)

Bringing back memories

Yesterday I didn't remember when I had exactly started using Debian. Tried hard but couldn't recall. I know for sure that the first version I installed was 3.0 Woody and shortly after Sarge 3.1 appeared. So today I have checked the history of Debian releases and it turns out that Woody appeared in 2002 and Sarge in 2005. This makes sense. As I told you yesterday I started learning linux in 2004 and became a full time user in 2005. So I moved from Woody to Sarge straight away.

The first linux version I ever tried and installed was S.u.S.E 9.2 professional. oh! It brings back so many memories ...

Five years using GNU/Linux

Yes!!!

As I have told my friends a million times I began fiddling with linux in 2004.  A year or so after buying 'Odd' I fell in love with it but it did not become my only true blue love until late 2005. I don't exactly remember when. Anyway it is now five years of giving and taking. That's what love is all about, ain't that so?

What I do not quite recall is when my beloved Debian came into sight. Too bad I don't remember one of the best moments of my life. But I think it didn't take long after discovering that rpm was not my thing.

What may the future bring? I do not know for sure. I think there will be a move to something new, something like GNU/Hurd. Debian GNU/Hurd for us diehard debian users.

Well I would like to explain this a bit further but by now I'm starving to death. I gotta have dinner ;-)

Freenas

freenas webgui on my eeepc

Freenas is an awesome piece of software. I have tried it in my network for some time and I love it.

Anyway I need to know it better to write more in depth.

Shutdown computer as a regular user (howto)

I sometimes need to shutdown one of my desktop machines after a long download. I was looking for a way to do it unattendedly. I found this curious method

In short:

1. Add these alias to .bashrc
alias halt="sudo /sbin/halt"
alias reboot="sudo /sbin/reboot"


2. Edit visudo
username ALL=(ALL) NOPASSWD: /sbin/halt, /sbin/reboot
(I actually had to edit it like this in my Lenny ALL=NOPASSWD: ALL)


3. Now I just need to make sure the download finishes to power off. I use something similar to this:
wget download ; shutdown -h now

Instead of that, if I were sure about the time of the download I could use the command like this:
shutdown -h + 45 (To delay it for 45 minutes) or shutdown -h 15:30 (To delay it until half past three pm)
That's it.

Let's automatize it!!!

I have devised a way to automatize the synchronization of files over ftp using lftp. I found the inspiration on linux.com as I told you in my previous post. I have written a here document like this:
#! /bin/bash
#script to automatize synchronization using lftp

lftp <<EOF
open -u user,password -p [port] [server]
mirror -c -e /remote_directory /local_directory
&&
open -u user,password -p [port] [server]
mirror -c -e -R /local_directory /remote_directory
exit
EOF

Now you know the drill:
I gave it execute permission: chmod +x
Then I put it in my $PATH (/usr/local/bin is a nice place to visit ;))
I added it to my crontab (or anacrontab if you prefer)

That's it!!! Now my directories synchronize automatically everyday. Maybe there is an ever better way to do it but that's the way I devised it and it works!!!

UPDATE: As usual there seems to be a better way to achieve this. I have found it here You can use 'at' with lftp:

lftp at 00:00 ... & (Thus the process runs in the background at 12)

FTP synchronization

I have found a very nice way of synchronizing local and remote directories over ftp using lftp and following this easy-to-understand article at linux.com

The procedure is quite simple. To download from the server you need to create a file named download.x and write (with your data):

open -u user,password -p [port] [server]
mirror -c -e /remote_directory /local_directory
exit

To upload files to the server you need to create another file named upload.x and write (with your data):

open -u user,password -p [port] [server]
mirror -c -e -R /local_directory /remote_directory
exit

To download files simply type 'lftp -f download.x' at the command line.
To upload files simply type 'lftp -f upload.x' at the command line.

That's it. Of course you can automatize these actions with cron or anacron.