root.log

Top wizardry

2006-05-06T14:44:00.000+02:00

OK, you're probably way too familiar with the Linux top command. But do you know what all of these mean?

top -b -n 1
top -u www-data
top -p 4365

If not, head over to this valuable article on using top more efficiently.

Backup all mysql databases to separate dump files

2005-10-11T13:40:00.000+02:00

It's easy to just simple do a

mysqldump --alldatabases > mysqlbackup.sql

to backup everything into a large SQL dump, so that you have the inserts there if you have a booboo happening. But what a nightmare you face when you just have to get those lines that affect your recently-crashed site! Well, there's a bit more sophisticated, but much better solution: have each of your database backed up in a separate SQL dump file with the following shell script:

for i in /var/lib/mysql/*/; do
dbname=`basename $i`
/usr/bin/mysqldump $dbname > /home/db_backups
done

Of course you might have to add login parameters to the mysql line above, but that shouldn't be a problem. (Hint: -u myusername --password=mysecretpassword)

Serve all - and any - subdomains from one central script

2005-10-02T16:27:00.000+02:00

This is probably the most common wildcard example, still, I needed some googling to find the perfect way to do it. The idea is that the setup should accept any kind of subdomains (we don't know in the beginning, what subdomains might come), and serve them from one central scritp. Of course all this without the user seeing what's going on behind the curtains. Let me make it clear with an example:

http://anything.example.com/somesubdir
should be served by
http://example.com/centralscript.cgi/anything/somesubdir

http://anotherthing.example.com/
should be served by
http://example.com/centralscript.cgi/anotherthing

etc.

So, as you see, we have a centralscript.cgi, which gets the subdomain name and the directories that were called as a parameter. This of course can be a PHP script too, whatever you prefer. Now, one more thing to the specification: www.example.com and the plain example.com should work as they are now, no centralscript.cgi magic here.

The solution was found at the wonderful webmasterworld forums - as so often. This topic discusses the exact same problem. I'll go into more details here and include the necessary nameserver configuration too, but I don't want to take credit for anything here, I would still be struggling without the fine guys at the webmasterworld.

So, first, let's set up our nameserver (bind9 is my choice, but whatever you use, this should work. I hope at least.), edit the zone file for your example.com domain. First of all, never forget to update the timestamp of the zone, so start with that, and then insert the following line to the end of your subdomain list:

* A 1.2.3.4

Where of course 1.2.3.4 is the IP of your domain name. Restart the nameserver (/etc/init.d/bind9 restart, or whatever you use), and check your setup with pinging anything.example.com. Provided that the test box uses the same nameserver too, you should get pretty fast response times. If not, maybe you should wait a bit until the domain zone file spreads to your test box - but be prepared for a bit more time than a cup of tea. Anyway, continue, if you can ping any subdomain under example.com - the nameserver part is done.

Next, Apache config and the mysterious rewrite engine. It's a topic worth to master, but honestly, I didn't have the time yet. The explanation of the techniques below can be found at the webmasterworld forums linked above.

So, open up your httpd.conf file, and look for the virtualhost already set up for your main example.com domain. (I won't explain it here how to do that, it's pretty basic.) Side-note: I'm very much against puting this kind of settings to .htaccess files. Only use those, if you absolutely don't have access to the central configuration file - just imagine, the httpd.conf files are loaded once you start the server, but the .htaccess files are loaded at every request. No need to overload your server without a reason.

First what you have to do is make sure that apache finds the appropriate virtualhost for all your subdomains. Add this line to the virtualhost settings:

ServerAlias *.example.com

To test if it works, restart apache, and go for http://whatever.example.com. You should see your original www.example.com main page, since no special rules are defined yet for this subdomain. If all goes well, open the httpd.conf file again, and paste this code below at the end of the virtualhost config of www.example.com:

# Rewrite <subdomain>.example.com/<path> to
#   example.com/centralscript.cgi/<subdomain>/<path> 
RewriteEngine On

# Skip rewrite if no hostname or if subdomain is www 
RewriteCond %{HTTP_HOST} . 
RewriteCond %{HTTP_HOST} !^www\. [NC] 

# Extract (required) subdomain (%1), and 
#   first path element (%3),
#   discard port number if present (%2) (in one line!)
RewriteCond %{HTTP_HOST}<>%{REQUEST_URI} 
    ^([^.]+)\.example\.com(:80)?<>/([^/]*) [NC] 

# Rewrite only when subdomain not equal to 
# first path element (prevents mod_rewrite recursion) 
RewriteCond %1<>%3 !^(.*)<>\1$ [NC] 

# Rewrite to /subdomain/path 
RewriteRule ^(.*) /centralscript.cgi/%1$1 [L]

# Do some logging
RewriteLog "/var/log/apache/rewrite.log"
RewriteLogLevel 9

Of course you'll have to replace example.com to your main domain, and centralscript.cgi for your script's name. Restart apache, and check if it works. If not, study the rewrite log set up above, if yes, comment out the two last lines - again, no need to overload the server with some useless logging junk.

I provide you a simple centralscript.cgi to test if the setup works, and also to see how the parameters are passed:

#!/usr/bin/perl
print "Content-type: text/plain\n\n";
print "CGI parameter (PATH_INFO): $ENV{'PATH_INFO'}";

Of course the usual blahblah applies: script should have the x flag, and ExecCGI should be allowed for the directory where it's in.

Common templates for different blogs in MovableType

2005-10-01T15:14:00.000+02:00

Say you have more than one (we have eight, and growing...) number of MT blogs that have templates that are the same for every one of them. (For example the RSS feed, individual archive, whatever.) You'd love to have them at a central place, so when it's time to finetune, you don't have to modify all the blogs' templates for the same changes. I hate doing repetitive stuff, so instead I found out that how I can avoid it.

MovableType supports a feature that lets you link a template to a file. Yes, that's going to be the solution: we link the same temaplate (in this example the RSS feed index.xml one) to the same file at every blog. The only problem is that the "link template to a file" input box's rootdir is the root of your blog, which is of course different for the different blogs.

For example you have this setup:

/home/blogs/blog1/ - www.blog1.com
/home/blogs/blog2/ - www.blog2.com

Now if you try to link to "/common_index.xml.tmpl" at blog1, it will create the file /home/blogs/blog1/common_index.xml.tmp. Which is of course hard to see from blog2. What you can do is set up a directory for all the common templates (for example: /home/blogs/common_templates) and link to it like this: ../common_templates/index.xml.tml

I know it's a dirty hack, it might even be a bug in the MovableType engine (but so far it's working at MT 3.2), but it works like a charm, you just have to set it up once at every blog, then a modification at any blog will affect all the other blogs too. You might need to rebuild all of your blogs to see it working in case you're not modifying a dynamic template.

To master this technique, you can separate common parts from individual ones with referring to template modules. If you refer to a module with <$MTInclude module="head"$> then every blog will look for the template module "head" in its own templates, so you can achieve individual heads, while having common index templates. Way to go!

Simple, automated server link check

2005-09-29T17:01:00.000+02:00

It's been ages since I've been looking for a solution to be able to check dead links on my sites. Just it never had a high priority - until now.

After some googling, installing, playing around, removing, googling, installing, etc... I've found the perfect script for me: CheckBot. The documentation is pretty much straightforward, but for a reference, I use this setup:

checkbot --verbose --cookies --file index.html --mailto my@addre.ss --dontwarn "(301|302|903|904|400|403)" --ignore "(feeds\.archive|jigsaw|https)" --sleep 0.2 http://www.site1.com http://www.site2.com

--verbose: this should be turned off for automatic running

--file index.html: that's because the report's in its own directory

--mailto: my@addre.ss: so that I see what was going on

--dontwarn "(301|302|903|904|400|403)": I pretty much only need the serious problems

--ignore "(feeds\.archive|jigsaw|https)": URLs with these strings I don't want to check, validators in the beginning, linked with another URL from every page, and I don't need https checking neither (I know the regexps could be better, but that was fine for me)

--sleep 0.2: Without this I experienced a sudden 1.2 load on the webserver, that's not nice. 0.2 seconds of delay between requests it OK for me - local files are still fast, and for remote files the network delay will be much longer anyway

And then the URLs at the end. Can be put to a crontab to run every week, and that's it.

Mysterious connection problem and resolution

2005-09-29T12:37:00.000+02:00

This was weird. 2 days ago my connection to a company IMAP server simply stopped working. To cut a long story short, after lots of trying and head-scratching, I could at least identify the problem at its roots: whenever I telneted to a working port of the IMAP server (it has webmail too, etc) I was faced with an empty screen, and after the first keypress, the connection dropped.

Now I use a Windows XP laptop through a wifi router for work, I don't want to get into details why, it's just given. I tried to locate the source of the problem: trying from a remote server works OK. Trying from connecting the laptop to the router instead of wifi: NOT OK. Trying from the home linux server connected to the router: OK. Trying from Knoppix on my laptop: OK.

Shit, the problem's in the networking part of Windows XP. Hell, all I need now is a reinstall... like I'd have time for that. But I had an idea: if the stuff works from my server, why don't I try to tunel the IMAP traffic through it? Why not - besides the fact that I've never done anything like that before.

But surprisingly it worked like a charm, thanks to this guide. You might want to turn on compression and ask putty not to open a remote shell, since you're not using the ssh connection itself, just for the tunel. When set up, I started the tunel, and asked my IMAP client to connect to localhost:143 - and voilá, after all the struggling, I have a faster and a bit more secure connection now. Phew.

Creating and using SFV checkfiles on debian

2005-07-13T10:49:00.000+02:00

This is quite straightforward, just I need it so rarely that I'm keep on forgeting, so here's the note. The package is cksfv, to create an sfv checkfile for sample.tgz use cksfv sample.tgz > sample.sfv, to check it just use cksfv sample.svf.

MovableType MTDateHeaders problem solved!

2005-07-11T15:26:00.000+02:00

I've updated the original article in case someone finds it through google. Phew.

Registered users cannot make modifications in MediaWiki

2005-06-07T13:24:00.000+02:00

This was bugging me for a while. There was a server restart and after that no registered users could make any changes in our MediaWiki install - unregistered users could write, change, whatever without any problems. I've updated to 1.4.5, no help. Finally I've turned to the IRC channel of MediaWiki - and a simple answer led me to the solution: "your php session support is broken. fix it."

I've checked my php.ini, and there I found:

session.save_path = /tmp/php4_sessions

Thinking a bit back I remember adding this subdirectory to the config a while ago, when I had enough of all the session files in the /tmp folder - what I forgot is that at restart, the contents of the /tmp folder is completely deleted - including subdirectories. Also, php warns me at the config that she'll not create subdirs for the sessions, so there it is - I've changed back the php.ini config to /tmp, and now everything works well.

p2p on debian

2005-05-14T11:16:00.000+02:00

As far as I know, MLDonkey is the best application for this: multiple protocols (edonkey, overnet, torrent, gnutella, etc.), telnet- and web-based client, dpkg installer, what else would you need? Just running it was not so obvious for me, so here it is:

Start daemon: mldonkey_server --daemon --start
Stop daemon: mldonkey_server --daemon --stop

I'm sure it should work normally by /etc/init.d, but it failed for me.

Debian Administration site

2005-05-05T08:57:00.000+02:00

This seems to be useful, tips and tricks site for Debian system administration. Bookmarked, too bad their rss feeds are not working in Opera 8.

Incoming FTP directory config

2005-04-24T19:16:00.000+02:00

I guess this task is quite usual, however, I've never done this before, and had to do some googling to make it work as I wanted.

Requirements:

user public should be able to create new files and subdirs under /home/joe/incoming/
only joe and nobody else should be able to read the contents of, rename and delete from /home/joe/incoming

After some finetuning, the addition to proftpd.conf is the following:

<Directory /home/joe/incoming>
 Umask   002     002
  <Limit STOR CWD MKD XMKD>
   AllowAll
  </Limit>
  <Limit READ RMD XRMD DELE SITE_CHMOD RNFR RNTO>
   AllowUser joe
   DenyAll
  </Limit>
</Directory>

Also, user joe is in the same group as user public.

Quite a boring post, I admit, but I want to save some time next time I have to do it.

screen for dummies ...

2005-04-24T14:02:00.000+02:00

... like me. screen is an oldschool, yet very useful package to handle multilple jobs. This post will not list all of its functions - which can be find almost everywhere - just serves as a virtual post-it note for myself with the for the most basic features.

Start a program in the background with screen:
```
screen -dmS sessionname prgname
```
- this will start prgname in a background screen named sessionname.
See what's going on in a background screen:
```
screen -r sessionname
```
- this will put you on the screen of the session named sessionname.
Put the foreground screen back to the background: CTRL+A and then D (for detach).

How to backup all your incoming and outgoing mails?

2005-04-23T15:47:00.000+02:00

It's so simple I'm wondering why I haven't thought of it before. GMail. Set up a new account, like yournamebackup@gmail.com. Add it to the automatic bcc: setting of your favourite email client(s). Set up an autoforward (keep mails on server) on your email host. (I go for the least work, so I just made a .forward file like this in my home directory:

myusername
mynamebackup@gmail.com

- so just a copy of my mails are forwarded to gmail.)

Voilá, you have 2 GBytes (and growing) backup of all your incoming and outgoing emails. Neat.

All kinds of file uploads to MediaWiki

2005-04-17T10:28:00.000+02:00

I'm using a MediaWiki install to work on the always-changing documentation of a current project. The default install doesn't allow file uploads at all. Here's what I had to do:

First, chmod a+w images in the wiki root directory. Now users can upload all kinds of files that are not blacklisted in MediaWiki. Unfortunately we need to store .doc Word documents too in the wiki, so I needed another step. After some googling I've found the info in an archive of the wikitech-list:

# If this is turned off, 
# users may override the warning for files not
# covered by $wgFileExtensions.
$wgStrictFileExtensions = true;

These lines are from the DefaultSettings.php file, but you're not advised to make changes there. Instead, just copy them to LocalSettings.php, and set the variable to false.

After this, uploads of any file format are immediately available to every user of the wiki. Since I'm controlling access at the Apache level, that's exactly what I wanted.

Moving a site

2005-04-15T08:50:00.000+02:00

I wake up in the morning, 8AM, very early for me, and what do I see? The site's DNS I expected to change during the night displays another user's site. This is crazy, the new server is supposed to host the site well, I've tested that yesterday night, so what's wrong? After a coffe and two cigarettes, I've found it:

The DNS got changed during the night as I expected, just TTL was quite high, and my laptop had the old address cached
But not the old server
And add to the chaos that apache got restarted on the old server too
Which meant that if somebody requested the site from the old server got another page, because the server didn't think he's supposed to serve pages for this domain
Which is exactly what happened to me
(And to a few other visitors too, unfortuantely)

Quick'n'dirty solution: I've lied to the old server that he's still in charge of this domain name by adding a line to /etc/hosts... After an apache restart, all's fine now, regardless of how fresh or how old your DNS record is. Now off to have a breakfast.

Understanding Linux memory issues

2005-04-12T16:37:00.000+02:00

Just found a nice support page at a hosting company on troubleshouting memory usage. Blognote to self: finetune apache and mysql configs!

Rescue / burn-in boot cds

2005-04-11T11:12:00.000+02:00

A friend of mine is having a new laptop today (the bastard) and I advised him to "burn-in" the beast before installing anything on it. And of course I didn't remember the name of the tool I use for this, so after some googleing I assembled this list, so next time I can just look it up here. (If I remember that I've posted it. Whatever.)

The Ultimate BootCD - all kinds of burn-in tests for RAM, CPU, HDD, etc.
KNOPPIX Linux LiveCD - there are thousands of different Linux LiveCDs out there now, but this I actually used before, and found out with great pleasure that it reads NTFS and burns CDs. Probably the others do too.
UBCD for Windows - Sideproject of The Ultimate BootCD, with a live windows (you'll need your install to build the cd) running from the cd itself. Sounds promising, but fortunately I never needed it yet.

Spambot = DOS

2005-04-11T09:39:00.000+02:00

This is getting serious. Yesterday night a spammer put 245 comments in less than 2 minutes to some MovableType blogs I host. The result: the server got overloaded in no time and the kernel started a great massacre in there. After the smoke cleared up crond, named and munin-node were killed - for the whole night, I just restarted them. It's really pissing me off, an automated process can bring down a server in 2 minutes, I'll have to do something about this. Ideas:

upgrade to MT3, check how it's done there
disable comments for old posts (is it possible in MT2.6?)
hack comments.cgi not to allow more than 1 comment / 1 minute / 1 IP
replace MTBlackList with a better protection tool

That's all I can think of. Together with the previous trouble with MT2.6 I think it might be time to upgrade.

Strange debian-mysql-apache-mod_perl problem

2005-04-10T14:18:00.000+02:00

This got me. I was adding a new script to an already working mod_perl site, and when I found it working on my Windows box, I "released" it to the live Linux server.

It didn't work. Without getting into too much details, I could summarize what caused the problem:

use DBI;
print "Content-type: text/html\n\n OK!";
print eval{ 
 DBI->connect("DBI:mysql:database=dbname",'user','pass', 
 { RaiseError => 1, PrintError => 1, AutoCommit => 1 }) 
};

The above script nicely connects through the command line, but when loaded into the browser, it returns 0 bytes and closes the connection. If I comment out the connection line(s), the whole thing works OK from mod_perl too.

The site's error log gets the following messages:

free(): invalid pointer 0x917cb90!

Strange. Add to the frustration that the other parts of the site use the database connection without a problem.

No happy end: I moved the new function under CGI, there it works.

Sometimes I really wish I was a baker or something.

Resizable textarea

2005-04-07T14:56:00.000+02:00

Just a reminder for myself, next time I'll need a textarea being able to contain lots of info, I should make it resizable. Or maybe just allow the user to edit the contents of the page.

Backslashes in php regular expressions

2005-04-06T13:39:00.000+02:00

Damn this was frustrating! Trying to match a \ character in php? Don't expect the usual Perl behavior, finding \\ - it won't work. You have to search for \\\ ! And the solution was only found in a source code with google:

  function deslash($content) {
    // Note: \\\ inside a regex denotes a single backslash.

    // Replace one or more backslashes followed by a single
    // quote with a single quote.
    $content = preg_replace("/\\\+'/", "'", $content);

    // Replace one or more backslashes followed by a double
    // quote with a double quote.
    $content = preg_replace('/\\\+"/', '"', $content);

    // Replace one or more backslashes with one backslash.
    $content = preg_replace("/\\\+/", "\\", $content);

    return $content;
  }

2 hours wasted...

Backing up an ftp-only site

2005-04-06T10:48:00.000+02:00

One of my new sites has an ftp account only, but still, I'm only sleeping well if I can back everything up. I thought a simple mirror would be enough, and when I have the files on a more controllable box, I can put them into my regular backups. That's exactly what I did:

Installed the mirror package with aptitude (by the way, it kicks the ass of apt-get)

Created the following mini package file, saved as /home/someserver_backup/someserver_mirror_package:

package=someserver.com
site=ftp.someserver.com
local_dir=/home/someserver_backup/someserver.com/
remote_user=someftpuser
remote_password=someftppassword
remote_dir=/
passive_ftp=true
get_patt=(^http|^cgi)

Tested it with mirror -n /home/someserver_backup/someserver_mirror_package (if you see filenames running on the screen, you're probably ok)

Added /usr/bin/mirror /home/someserver_backup/someserver_mirror_package to backup2l.conf's PRE_BACKUP section

Ran backup2l -b to see if everything's ok

Smiled

MovableType monthly archive problems

2005-04-06T09:03:00.000+02:00

I just moved my MT blog to a new server, and found that the monthly archive page does some weird stuff. </MTMonthHeader> appearing in the rendered page's source was a sign that one of the plugins (MTDateHeaders to be specific) is not working. Of course, no errors during rebuilding.

I rememberd a similar issue with AWStats, where after all it turned out that one of the plugins were missing a perl module. Some googling showed that this module uses Date::Calc, and I didn't have it on the new box! Until now, at least...

But no help, this module still doesn't work. And - maybe it's just me, but - I cannot find the original plugin to download and check what else it misses. I gave up here yesterday night, today I might just update the whole MT package to 3.15, and find out how to do the MonthHeader thing differently.

Update: I've recently upgraded to the latest MovableType (3.17), with a clean install, but the problem was still there. OK, this time hardcore googling. Finally I've found a forum post where the author of the MTDateHeaders explained some install issues. One thing stroke me: he was referring an MTDateHeaders.pm file too, together with the usual pl. Sparks above head. That's going to be it!

Fortunately I still had access to the old server where everything was working ok. Did a quick'n'dirty find:

find / -name MTDateHeaders.pm

And there it was, this nasty, bad bad bad module under /usr/local/share/perl/5.8.3! Moved to the new server, and there it goes, everything works nicely.

Before I finish, let me summarize: there's a software on the server, MovableType. There's a small plugin for it, MTDateHeaders. MT has a nice extlib directory for all the modules she and the plugins might need. Added all these together, a question rises: How the hell dares this plugin touch the base perl library directory?! And if it's unavoidable (it is not), why cannot he complain if his vital module is missing?! Wonders of the world, I guess.

Restarting updated services

2005-04-05T09:10:00.000+02:00

This script posted to the debian security list by Frans Pop seems to be very useful. It checks for services that are depending on recently updated libraries, and therefore should be restarted. Now I just have to remember to restart the whole server whenever there's a new kernel installed.

Update: I've found out that the debian-goodies package contains a script called checkrestart which does exactly the same. Well, now I know.