Categories
Computers Gadgets Linux

Dlink DNS-323, part 5 – The results

I have written some posts on the issues I have had over the past week with my Dlink DNS-323 NAS. As I wrote yesterday I configured the system to use JBOD to combine the size of the two Samsung 250GB disks and then fill the entire array with data to verify that I would be able to recover the files from one of the disks if the other failed.

The findings are interesting. Please read on.

Once both disks were completely full I powered down the DNS-323 and connected one disk at a time to a computer running Ubuntu 8.04, using a SATA-to-USB adapter.

The leftmost disk showed the following partitions:

Device Start End Blocks ID System
/dev/sdb1 1 66 530113+ 82 Linux swap / Solaris
/dev/sdb2 131 30401 243151807+ 83 Linux
/dev/sdb4 67 130 514080 83 Linux

However, the partition /dev/sdb2 would not be mounted. The error in the log was “VFS: Can’t find an ext2 filesystem on dev sdb2”.

I then switched to the other disk (the one on the right). The partition table looked identical but this time I could actually mount the disk – but whenever I ran ‘ls’ I got a lot of errors saying “cannot access test/D0000220: Input/output error”. The error was caused by files that were on the other disk but were referenced by the file allocation table on this disk. The files that were on this disk were accessible, however.

I then studied what data had been saved on which disk and visualised it. In the image below, each pixel represents two files, each 1MB each in size. The first saved file is the one in the top left and it then goes across and down. Red pixels were saved to the left disk whereas blue pixels were saved on the disk on the right. The original image was 1000 pixels wide, I just shrunk it horizontally to be able to fit within the boundaries of this blog.

The fact that it seems very difficult to recover files on one of the disks means that I will probably stay clear of both JBOD and RAID for my DNS-323. Too bad.

Categories
Computers Gadgets Mac

Dlink DNS-323, part 4 – Gotchas for Mac users

Here are some quick hints that made me lose a couple of hours when setting up the DNS-323 from an iMac:

  • Do not use Safari to access the web administration pages on DNS-323. Especially not when formatting the drives. It just stops at 94% and sits there. Use Firefox instead.
  • Do not use Cyberduck to transfer the fun_plug files to the DNS-323. For some reason, the NAS does not run the fun_plug file when it boots. Instead, use the console FTP client from a terminal to upload the files.
Categories
Computers Gadgets Linux

Dlink DNS-323, part 3 – Fonz fun_plug

This is a continuation of a previous post.

OK, the idea wasn’t exactly brilliant. The script worked fine but completely filling the disks (totally 500GB) from another computer was going to take up towards 40 hours and I am a little impatient. The bottleneck with the DNS-323 is as usual the network connection. So instead, I though about running the script on the DNS-323 itself – which should be very much quicker. But to do that I needed to install a fun_plug to be able to log on and run some software on it.

I had done some small tests with fun_plugs when I first got the DNS-323 but I haven’t checked how much could be done and I was pleasantly surprised. This is a step-by-step description on how to install Fonz fun_plug (FFP) and make it accessible through SSH.

  1. Download fun_plug and fun_plug.tgz from this web site
  2. Copy the files to the Volume_1 folder on the DNS-323
  3. Make sure that the fun_plug file is executable
  4. Restart the DNS-323, then telnet to get shell access
  5. Install all packages as described in the readme for FFP

    6. cd /mnt/HD_a2
    rsync -av inreto.de::dns323/fun-plug/0.5/packages .
    cd packages
    funpkg -i *.tgz

  6. Enable the root password and set a password as well as the shell for root by issuing the following commands

    7. pwconv
    passwd
    usermod -s /ffp/bin/sh root

  7. Verify that you can log on as root

    8. login

  8. If the login worked, then store the password to flash memory by running

    9. store-passwd.sh

  9. Start the ssh server (which will take a while since it has to create keys), then try to log on from another computer

    10. cd /ffp/start
    sh sshd.sh start

  10. If that worked it is time to turn off the telnet server and to enable the ssh server instead

    11. chmod a-x telnetd.sh
    chmod a+x sshd.sh

I am now running the script on the DNS-323 and it is about 7 times quicker than running it via Samba.

More to follow…

Categories
Computers Gadgets Linux

Dlink DNS-323, part 2 – JBOD vs separate disks

This is a continuation of a previous post.

With all my important data on another disk it was finally time to upgrade the DNS-323 to the newest firmware and to reformat the disks. This also brought up the question whether I should use JBOD or separate disks. After searching on the Internet, there seems to be a lack of evidence of just how the DNS-323 handles disks in a JBOD array. And so I wondered if maybe I should test and document it.

To do that I wrote the following little Bash script and run against the JBOD array from another computer. The script creates numbered files, each with a size of 1MB. 1000 such files are placed in each directory. The plan was to fill the entire disk and then take out the disks to study and see what the DNS-323 had stored on each disk and to verify that the content on a disk would in fact be accessible if the other disk broke down.

#!/bin/bash
 
TARGET=/mnt/nas
FOLDERS=500
FILES_PER_FOLDER=1000
BLOCKS_PER_FILE=2000
 
if [ ! -d $TARGET ]; then
  echo "Target folder does not exist"; exit
fi
 
for d in `seq 1 $FOLDERS`; do
  dirname=`printf 'D%07d' $d`
  echo "Creating folder: $dirname"
  mkdir $TARGET/$dirname
 
  echo " Creating file: F0000001"
  dd if=/dev/zero of=$TARGET/$dirname/F0000001 count=$BLOCKS_PER_FILE
  for f in `seq 2 $FILES_PER_FOLDER`; do
    filename=`printf 'F%07d' $f`
    echo " Copying file: $filename"
    cp $TARGET/$dirname/F0000001 $TARGET/$dirname/$filename
  done
 
done

Please check back for the result of these tests.

Categories
Computers Gadgets Linux Mac Windows

Filename encoding problems on Dlink DNS-323

I have had my Dlink NAS DNS-323 since early 2007. It has mostly served me well. Over the months I have put more and more files on it so that it now holds about 350GB of data. Out of fear of losing precious data I have not updated the firmware so I am still on 1.03 from May 2007.

I mounted a shared folder on the DNS-323 from a Ubuntu client and noticed that the Swedish characters were all messed up. First I thought the error was related to how I mounted the drive from Linux, but then I found out that the issue is with the DNS-323 itself and the fact that it uses a non-Unicode character set for the filenames. This should be solvable with the iocharset and the codepage parameters to the mount command in Linux but I couldn’t get it to work.

Later firmwares are said to fix the problem – but only if the drives are totally wiped. I got myself a USB drive sufficiently large to hold everything and copied all the data over using rsync so now I am just about ready to upgrade the firmware and reformat the drives and use some of the plugins on http://wiki.dns323.info. But more on that some other time.

Before I wipe the disks I wanted to make sure that I could rename all the files using Unicode but with some 50,000 files I didn’t want to do it manually. The Linux command iconv can convert between encodings but it works on a file level and I wanted something that only touches the filenames, not the contents of the files.

I found the Perl command convmv which is available through the standard Ubuntu repositories. Just type “apt-get install convmv”. It does the same as iconv but on filename level. Precisely what I needed. I then typed:

#/mnt/wd640gb# convmv -f cp850 -t utf8 -r .

This command shows how files would be renamed, switching from codepage CP850 (the default or DNS-323) to UTF8. Once you are happy with the suggestions, just issue the command again but with the extra switch –notest to actually rename the files.

My only issue now is that convmv only works on filenames, not directories. But at least I have reduced by problem by a factor 30 or something. The directories I can do manually.

Categories
Computers Gadgets

Thought about writing iPhone apps – now less sure

I have been thinking about doing some development for iPhone and have come up with some really cool application ideas that I would like to turn into real applications. However, after reading on various blogs I have realised that the limitations in the development environment on the iPhone mean that those applications are not possible; at least not in a way that would allow me to distribute them through AppStore. That, of course, means that they are dead in the water since the vast majority of iPhone users will only use the AppStore.

What is holding me back? Two words – background processes.

The sad story is that third-party applications can not be made to stay running in the background when the user switches task, takes a call or when the phone goes into sleep mode. Of course, this limitation does not apply to Apple’s own applications. The argument that Apple is pushing is that they need to ensure that background processes don’t slow down the phone or drain the battery. This may seem valid but personally I think it should be left for the end user to decide. It could even be done so that the AppStore clearly states how long the application may be running in the background before it is installed – after which time the OS shuts it down.

I hope this is fully addressed by Apple soon – and no, the promised notification service is nowhere near solving my problem. Otherwise I may go back to developing my applications for Symbian or Windows Mobile – both of which I have written applications for in the past.

Categories
Computers Linux

Adding roles to Ubuntu Server

I have set up a number of server based on the past four or five versions of Ubuntu Server. Every time I face the dialog where one can pick what additional software to install, I just select OpenSSH so that I can make the server headless and manage it remotely. I then install each individual package using apt-get until the system works as I wish.

Ubuntu 8.10 Server - additional software roles

First, I thought that the alternatives for software could be installed as meta packages using apt-get but I never managed to find any such meta packages in the package list on http://packages.ubuntu.com – or by searching the repositories with apt-cache.

I just now learned that the command tasksel brings up a menu similar (but extended) to the one shown during the installation. It is also possible to list which deb packages an individual task would install. A simple method would for instance be to run ‘tasksel install dns-server’ which would lead to the exact same result as if one had picked the DNS server from the beginning.

Categories
Computers Linux

Keeping track of installed apps, part 2

In an earlier post I wrote how to output a list the currently installed applications under Debian (and distributions based upon it). This will now be extended into a script that can be run each hour to record changes to the installed applications. Now, it should be said from the beginning that this information may already be stored in the file /var/log/apt/term.log but it can be a challenge to follow what it happening.

The following script stores a snapshot of the installed applications and then makes a diff against that on a period basis (e.g. by running it as an hourly cron job). If there is a difference, the changes are saved to a time stamped file and a new snapshot is taken. All files are placed in the same log directory as apt normally uses (i.e. /var/log/apt). The downside of this method is that changes will be recorded with a granularity of an hour but usually that is not an issue as the reason for writing this was to keep an automated record of changes to the system.

#!/bin/bash
 
folder='/var/log/apt/'
installed=$folder'current'
 
if [ ! -e $installed ]; then
  echo "Creating initial file"
  dpkg-query -W -f='${Package}\n' > $installed
  cp $installed $folder'initial'
fi
 
# Compare package list against current
dpkg-query -W -f='${Package}\n' | diff $installed - \
  | grep -e '^[(<|>)]' > /dev/null
 
if [ $? -eq 0 ]; then
  # The set of installed packages has changed. Save the delta to a
  # file and save the new snapshot
 
  filename=$folder`date +%Y-%m-%d_%H-%M-%S`
  dpkg-query -W -f='${Package}\n' | diff $installed - \
    | grep -e '^[(<|>)]' > $filename
  dpkg-query -W -f='${Package}\n' > $installed
fi;
 
exit 0

Categories
Computers

Simple localisation of Rails validation messages

Rails scaffolding can be good to get up running quickly but it can also be a problem. This is for instance true when it comes to localising the error messages presented by Rails (e.g. “X errors prohibited this <object> from being saved”). The issue is caused by two things:

  1. Rails includes English default message headers for the error box without being apparent how to change them.
  2. Each individual error is prefixed by the capitalised field name. Regardless of GUI language these will probably be in English.

The first of these issues can easily be solved by replacing the default <%= f.error_messages %> with something like:

<%= error_messages_for :item,
:header_message => "Försök igen",
:message => "Några av värdena du har angivit är inte giltiga:" %>

The second is a little harder but can be solved by realising that Rails calls the method human_attribute_name on the model object, passing the column name. While it would be possible to do a switch statement or a lookup array to translate the column name, I have found it easier to just return an empty string and to provide the exact error message as part of the validation instead.

validates_presence_of :name, "Namnet måste anges"


def self.human_attribute_name(attr)
return ""
end

Categories
Computers

Boolean portability under Rails

Booleans should be so simple. They can only have two values. As a former colleague used to say – How hard can it be? With all the variants of “0/1”, “Y/N”, “t/f” the answer seems to be – Apparently quite so.

Rails 2.0 switched the default database engine to sqlite3 and this has now caused me two issues. The first I wrote about some time back. The last one I just stumbled upon was related to booleans.

This works in MySQL:

@guests = Guest.find(:all, :conditions => 'accept = 1')

But it fails in sqlite3 since Rails stores booleans as ‘t’ or ‘f’ on that database engine. The portable way to write the condition in Rails is to make Rails itself select the proper value depending on the underlying database by writing it like this:

@guests = Guest.find(:all, :conditions => ['accept = ?', true])

css.php