Categories
Computers Gadgets Linux

Dlink DNS-323, part 2 – JBOD vs separate disks

This is a continuation of a previous post.

With all my important data on another disk it was finally time to upgrade the DNS-323 to the newest firmware and to reformat the disks. This also brought up the question whether I should use JBOD or separate disks. After searching on the Internet, there seems to be a lack of evidence of just how the DNS-323 handles disks in a JBOD array. And so I wondered if maybe I should test and document it.

To do that I wrote the following little Bash script and run against the JBOD array from another computer. The script creates numbered files, each with a size of 1MB. 1000 such files are placed in each directory. The plan was to fill the entire disk and then take out the disks to study and see what the DNS-323 had stored on each disk and to verify that the content on a disk would in fact be accessible if the other disk broke down.

#!/bin/bash
 
TARGET=/mnt/nas
FOLDERS=500
FILES_PER_FOLDER=1000
BLOCKS_PER_FILE=2000
 
if [ ! -d $TARGET ]; then
  echo "Target folder does not exist"; exit
fi
 
for d in `seq 1 $FOLDERS`; do
  dirname=`printf 'D%07d' $d`
  echo "Creating folder: $dirname"
  mkdir $TARGET/$dirname
 
  echo " Creating file: F0000001"
  dd if=/dev/zero of=$TARGET/$dirname/F0000001 count=$BLOCKS_PER_FILE
  for f in `seq 2 $FILES_PER_FOLDER`; do
    filename=`printf 'F%07d' $f`
    echo " Copying file: $filename"
    cp $TARGET/$dirname/F0000001 $TARGET/$dirname/$filename
  done
 
done

Please check back for the result of these tests.

Categories
Computers Gadgets Linux Mac Windows

Filename encoding problems on Dlink DNS-323

I have had my Dlink NAS DNS-323 since early 2007. It has mostly served me well. Over the months I have put more and more files on it so that it now holds about 350GB of data. Out of fear of losing precious data I have not updated the firmware so I am still on 1.03 from May 2007.

I mounted a shared folder on the DNS-323 from a Ubuntu client and noticed that the Swedish characters were all messed up. First I thought the error was related to how I mounted the drive from Linux, but then I found out that the issue is with the DNS-323 itself and the fact that it uses a non-Unicode character set for the filenames. This should be solvable with the iocharset and the codepage parameters to the mount command in Linux but I couldn’t get it to work.

Later firmwares are said to fix the problem – but only if the drives are totally wiped. I got myself a USB drive sufficiently large to hold everything and copied all the data over using rsync so now I am just about ready to upgrade the firmware and reformat the drives and use some of the plugins on http://wiki.dns323.info. But more on that some other time.

Before I wipe the disks I wanted to make sure that I could rename all the files using Unicode but with some 50,000 files I didn’t want to do it manually. The Linux command iconv can convert between encodings but it works on a file level and I wanted something that only touches the filenames, not the contents of the files.

I found the Perl command convmv which is available through the standard Ubuntu repositories. Just type “apt-get install convmv”. It does the same as iconv but on filename level. Precisely what I needed. I then typed:

#/mnt/wd640gb# convmv -f cp850 -t utf8 -r .

This command shows how files would be renamed, switching from codepage CP850 (the default or DNS-323) to UTF8. Once you are happy with the suggestions, just issue the command again but with the extra switch –notest to actually rename the files.

My only issue now is that convmv only works on filenames, not directories. But at least I have reduced by problem by a factor 30 or something. The directories I can do manually.

Categories
Computers Linux

Adding roles to Ubuntu Server

I have set up a number of server based on the past four or five versions of Ubuntu Server. Every time I face the dialog where one can pick what additional software to install, I just select OpenSSH so that I can make the server headless and manage it remotely. I then install each individual package using apt-get until the system works as I wish.

Ubuntu 8.10 Server - additional software roles

First, I thought that the alternatives for software could be installed as meta packages using apt-get but I never managed to find any such meta packages in the package list on http://packages.ubuntu.com – or by searching the repositories with apt-cache.

I just now learned that the command tasksel brings up a menu similar (but extended) to the one shown during the installation. It is also possible to list which deb packages an individual task would install. A simple method would for instance be to run ‘tasksel install dns-server’ which would lead to the exact same result as if one had picked the DNS server from the beginning.

Categories
Computers Linux

Keeping track of installed apps, part 2

In an earlier post I wrote how to output a list the currently installed applications under Debian (and distributions based upon it). This will now be extended into a script that can be run each hour to record changes to the installed applications. Now, it should be said from the beginning that this information may already be stored in the file /var/log/apt/term.log but it can be a challenge to follow what it happening.

The following script stores a snapshot of the installed applications and then makes a diff against that on a period basis (e.g. by running it as an hourly cron job). If there is a difference, the changes are saved to a time stamped file and a new snapshot is taken. All files are placed in the same log directory as apt normally uses (i.e. /var/log/apt). The downside of this method is that changes will be recorded with a granularity of an hour but usually that is not an issue as the reason for writing this was to keep an automated record of changes to the system.

#!/bin/bash
 
folder='/var/log/apt/'
installed=$folder'current'
 
if [ ! -e $installed ]; then
  echo "Creating initial file"
  dpkg-query -W -f='${Package}\n' > $installed
  cp $installed $folder'initial'
fi
 
# Compare package list against current
dpkg-query -W -f='${Package}\n' | diff $installed - \
  | grep -e '^[(<|>)]' > /dev/null
 
if [ $? -eq 0 ]; then
  # The set of installed packages has changed. Save the delta to a
  # file and save the new snapshot
 
  filename=$folder`date +%Y-%m-%d_%H-%M-%S`
  dpkg-query -W -f='${Package}\n' | diff $installed - \
    | grep -e '^[(<|>)]' > $filename
  dpkg-query -W -f='${Package}\n' > $installed
fi;
 
exit 0

Categories
Computers Linux

Keeping track of installed apps under Debian

I am setting up my new server with Ubuntu 8.10. This time, I am planning to keep track on all the software I install on it so that I might restore it in the future. Thankfully, with the Debian package manager this is a simple task.

The following command creates a text file with all the installed applications on the system:

dpkg-query -W -f='${Package\n}' > installed-apps

The file will put one one package name on a line and end it with a line break, saving the output to the file installed-apps. By changing the \n at the end into a space it is possible to get a single long string with all the package names.

After a while you may want to know which packages have been installed since the initial operating system installation. There are no doubt many ways of doing this but the easiest I have come up with is to run the same command as above, but save the output to another file, and then to run ‘diff’ on those two files. Note that this requires that the output from the dpkg-query command above was using line breaks to separate packages.

Categories
Computers Gadgets Linux

Automatic boot after power loss on Asus Eee Box

I have been planning to upgrade my old Mini-ITX based Linux server with an Asus Eee Box. I have purchased the Eee Box and was just waiting for the upcoming release of Ubuntu 8.10 to run on it. But then I noticed that the computer didn’t start automatically after a power outage. There is usually a setting in the BIOS for this but I just couldn’t find it on the Eee Box. Not being able to boot automatically after a power loss posed a major setback for my plan of migrating my server.

It turns out that the Eee Box BIOS prior to version 0902 didn’t have support for restore after power-loss so to solve the issue I had to upgrade the BIOS.

Normally, BIOS upgrades can be a source for some concern as there is always a risk that the computer is left completely bricked. Also, it can be difficult to get a bootable USB device with the correct BIOS flashing utility and with the new BIOS firmware.

The BIOS firmware can be downloaded from Asus support pages by searching for the Eee Box model name (B202).

The documentation for the Eee Box was not very helpful. It said that there is a upgrade utility in the OEM version of Windows that the computer came with. However, the first thing I had done with the computer was to wipe the hard drive and to install Ubuntu.

The solution to the problem is to insert a FAT-16 formatted USB device with the desired BIOS firmware, then boot while pressing Alt+F2 to get into a special boot menu. There, one can choose the firmware file.

This was definitely the simplest BIOS upgrade processes I have come across – once I found the information on how to do it.

Categories
Computers Gadgets Linux

Upgrading the RAM on Eee Box

As I wrote in a previous post, I recently purchased an Eee Box. As I targeted it as a replacement mail and web server I thought that the included 1GB RAM was a tad low. I bought the replacement memory together with the Eee Box itself and here are some images and comments outlining the process of upgrading the memory.

Both the hard drive and the memory are accessible from the bottom of the device. Removing the table stand (notice the screw mount on the left in the image) reveals the following:

Before RAM upgrade

The left one of the two Eee Box stickers needs to be removed in order to access the hard drive. The sticker on the right needs to be removed for the RAM upgrade. Although I was only planning on upgrading the RAM, I peeled off both stickers.

Without stickers and with the hard drive removed

The hard drive is a Seagate Momentus 5400.5 160GB. It looks simple enough to switch out for a larger one if that is required. 160GB is plenty for the tasks that I will use it for so I didn’t change it.

Original Eee Box hard drive

With the two screws on the right (see image above) removed the side cover can be removed. This led to some confusion as the few other guides on the Internet didn’t mention exactly how to pry off the cover. To help you out, have a look on the following image. The side where the screws were (shown with red) is already loose and there is no need to try to start from there. Instead take a table knife and work on the spots shown with the green crosses where the cover is kept in place.

How to open the Eee Box

With the side cover off we now need to unscrew just one screw in order to access the memory compartment in the lower right:

Eee Box with the side cover off

The unit was originally equipped with two DDR2 667 MHz SO-DIMM cards of 512MB each.

The original RAM memory is exposed

I took out the original memory and instead inserted two 1GB DDR2 667 MHz SO-DIMM cards made by Crucial.

New RAM memory

Finally, I booted the computer and checked the memory.

Task manager showing 2GB of RAM

Five minutes after this I had started installing Ubuntu 🙂

While this process was slightly more complex than changing memory on just about any other computer (laptop or stationary) some kudos goes to Asus for making it much easier to change the RAM than what was the case on my Acer Aspire One. The only really non-trivial part was getting the side cover off without breaking it. However, once I knew how to open it, it was actually trivial as well.

Categories
Computers Gadgets Linux

Uuunboxing the Eee Box

The Eee Box has finally been launched here in Sweden. I have had my eyes on it for some time as I am in dire need of upgrading my old server (mail, web etc) and this looks ideal from a performance/electricity point of view. It also comes at a very good point with the rather recent LTS release of Ubuntu 8.04 – although I will probably go for the bleading edge and run Intrepid Ibex when it is released on October 30th. The Eee Box is only available with Windows here in Sweden but hopefully that will change over time.

Here are some unboxing images:

The box from the outside
The box from an angle
Revealing the Eee Box
The Eee Box
Keyboard
Power and mouse
Documentation
PSU
Stand and mount
Mouse
Antenna and screws
Manuals and discs
All contents
Eee Box on its stand
Size comparison with iPhone
Rear connectors
Front connectors
BIOS main screen
BIOS advanced screen
BIOS power screen
BIOS boot screen
BIOS tools screen
BIOS exit screen

Categories
Computers Linux

Proxy a web site and change the content on the fly

Apache can do just about everything with a bit of configuration. I have used HTTP proxying on a number of occasions to make content from one site appear to come from another site. This can be very handy for sites I host at home where I only have one IP address but two servers and want to host different web sites on the same public IP – and all of them on the standard HTTP port.

However, until recently I hadn’t experienced that Apache can also rewrite the content in the actual response. I was faced with the requirement to change all the URIs in the proxied web site so that they pointed to the new URL. When I first searched the net I found the module mod_proxy_html which sounded like the way forward. In the end I didn’t go that route since I found it overly complex and didn’t transform all instances of the links. Instead, the solution was simple and used another Apache module – mod_substitute.

The following virtual host configuration sets up a proxy so that the site www.one.com is proxied to www.two.com. In addition, any occurrences of www.one.com is changed into www.two.com:


<VirtualHost *>
  ServerAdmin postmaster@two.com
  ServerName www.two.com
  <Proxy *>
    Order deny,allow
  </Proxy>
  ProxyRequests on
  ProxyPass / http://www.one.com/
  ProxyPassReverse / http://www.one.com/
  AddOutputFilterByType SUBSTITUTE text/html
  Substitute "s|www.one.com|www.two.com|n"
</VirtualHost>

Categories
Computers Linux

Can’t add column in sqlite3 on Mac OS X 10.4

Mac OS X 10.4 ships with sqlite3 3.1.3 which apparently doesn’t support ALTER TABLE in order to add columns to existing tables. At least that was the explanation I found on why I couldn’t add a migration in a Rails project to add a column. Instead I got the following error when I ran “rake db:migrate”:


== 20081007233057 AddIngredientToRecipe: migrating ================================
-- add_column(:recipes, :ingredient_id, :integer)
rake aborted!
SQLite3::SQLException: near "ADD": syntax error: ALTER TABLE "recipes" ADD "ingredient_id" integer

To fix the problem I had to run “sudo gem install sqlite3-ruby”. Before I got that far, I upgraded my sqlite3 installtion to the latest version by compiling from source. I don’t know if that was actually necessary in order to solve the issue.

css.php