When installing a Ubuntu 8.04.1 virtual guest under VirtualBox 2.06 running on a Mac you will probably be faced with the following error:
Starting up ...
This kernel requires the following features not present on the CPU:
0:6
Unable to boot - please use a kernel appropriate for your CPU.
This problem is due to the fact that the last couple of versions of Ubuntu have been compiled with PAE enabled – but the default guest setting of VirtualBox is that PAE is disabled. So to solve the issue, just stop the virtual guest and enable support in the CPU for PAE/NX, it’s under the advanced general settings. Another solution would be to reinstall the guest using the alternate CD image of Ubuntu (which, last time I checked, didn’t require PAE).
Category: Computers
I have had my Canon EOS 20D since early 2005. The first few weeks I set the camera to save images in JPEG only. After a while I switched over to saving images in both JPEG and RAW and have been doing so up until this year. Up until last year I had been using Windows XP and Windows Vista and having JPEGs around made it easier to look at the photos. However, about a year ago I switched over to Mac and am now using Aperture 2 for my photo cataloging needs. There, the presence of both JPEGs and raw images is nothing but annoying.
To avoid the problem with both formats in Aperture I want to import the raw images where available and JPEGs otherwise. But I can’t just remove the JPEG files on a folder level because some images are only available as JPEGs. And with literally tens of thousands of images I just didn’t want to do it manually.
The attached Perl script solves the issue. It takes a source and a target folder as arguments. It then goes through the source directory hierarchy and copies all the image files to the target – but skipping files that are available both as RAW and JPEG. In that case it will pick the RAW file. It uses embedded EXIF tags (the time the photo was taken plus the serial number of the image) to judge if two images are the same. Further, it retains the folder structure but removes certain folders to flatten the target folder structure – I had originally put the RAW files one folder down so that they wouldn’t interfere with the JPEGs when viewing them in Vista’s image viewer.
Please note that I can only vouch that this works on CR2 files and JPEG files from a Canon EOS 20D as that is the only thing I have tested it with. It should be simple to adapt it for other cameras. Also note that the script does not test whether the target folder is empty. You are advised to test the program on some files that you don’t mind losing before you apply it to your entire image library.
I called the script photo_prune, despite the fact that it doesn’t actually prune the source data. To avoid data loss it instead copies the data to a new location.
The image in the previous post was made through a small Perl script and using GD. I had saved a list of files on one of the disks in my DNS-323 NAS and wanted to visualise it. The Perl code for this was this:
I have written some posts on the issues I have had over the past week with my Dlink DNS-323 NAS. As I wrote yesterday I configured the system to use JBOD to combine the size of the two Samsung 250GB disks and then fill the entire array with data to verify that I would be able to recover the files from one of the disks if the other failed.
The findings are interesting. Please read on.
Once both disks were completely full I powered down the DNS-323 and connected one disk at a time to a computer running Ubuntu 8.04, using a SATA-to-USB adapter.
The leftmost disk showed the following partitions:
| Device | Start | End | Blocks | ID | System |
| /dev/sdb1 | 1 | 66 | 530113+ | 82 | Linux swap / Solaris |
| /dev/sdb2 | 131 | 30401 | 243151807+ | 83 | Linux |
| /dev/sdb4 | 67 | 130 | 514080 | 83 | Linux |
However, the partition /dev/sdb2 would not be mounted. The error in the log was “VFS: Can’t find an ext2 filesystem on dev sdb2”.
I then switched to the other disk (the one on the right). The partition table looked identical but this time I could actually mount the disk – but whenever I ran ‘ls’ I got a lot of errors saying “cannot access test/D0000220: Input/output error”. The error was caused by files that were on the other disk but were referenced by the file allocation table on this disk. The files that were on this disk were accessible, however.
I then studied what data had been saved on which disk and visualised it. In the image below, each pixel represents two files, each 1MB each in size. The first saved file is the one in the top left and it then goes across and down. Red pixels were saved to the left disk whereas blue pixels were saved on the disk on the right. The original image was 1000 pixels wide, I just shrunk it horizontally to be able to fit within the boundaries of this blog.
The fact that it seems very difficult to recover files on one of the disks means that I will probably stay clear of both JBOD and RAID for my DNS-323. Too bad.
Here are some quick hints that made me lose a couple of hours when setting up the DNS-323 from an iMac:
- Do not use Safari to access the web administration pages on DNS-323. Especially not when formatting the drives. It just stops at 94% and sits there. Use Firefox instead.
- Do not use Cyberduck to transfer the fun_plug files to the DNS-323. For some reason, the NAS does not run the fun_plug file when it boots. Instead, use the console FTP client from a terminal to upload the files.
This is a continuation of a previous post.
OK, the idea wasn’t exactly brilliant. The script worked fine but completely filling the disks (totally 500GB) from another computer was going to take up towards 40 hours and I am a little impatient. The bottleneck with the DNS-323 is as usual the network connection. So instead, I though about running the script on the DNS-323 itself – which should be very much quicker. But to do that I needed to install a fun_plug to be able to log on and run some software on it.
I had done some small tests with fun_plugs when I first got the DNS-323 but I haven’t checked how much could be done and I was pleasantly surprised. This is a step-by-step description on how to install Fonz fun_plug (FFP) and make it accessible through SSH.
- Download fun_plug and fun_plug.tgz from this web site
- Copy the files to the Volume_1 folder on the DNS-323
- Make sure that the fun_plug file is executable
- Restart the DNS-323, then telnet to get shell access
- Install all packages as described in the readme for FFP
- Enable the root password and set a password as well as the shell for root by issuing the following commands
- Verify that you can log on as root
- If the login worked, then store the password to flash memory by running
- Start the ssh server (which will take a while since it has to create keys), then try to log on from another computer
- If that worked it is time to turn off the telnet server and to enable the ssh server instead
cd /mnt/HD_a2
rsync -av inreto.de::dns323/fun-plug/0.5/packages .
cd packages
funpkg -i *.tgz
pwconv
passwd
usermod -s /ffp/bin/sh root
login
store-passwd.sh
cd /ffp/start
sh sshd.sh start
chmod a-x telnetd.sh
chmod a+x sshd.sh
I am now running the script on the DNS-323 and it is about 7 times quicker than running it via Samba.
More to follow…
This is a continuation of a previous post.
With all my important data on another disk it was finally time to upgrade the DNS-323 to the newest firmware and to reformat the disks. This also brought up the question whether I should use JBOD or separate disks. After searching on the Internet, there seems to be a lack of evidence of just how the DNS-323 handles disks in a JBOD array. And so I wondered if maybe I should test and document it.
To do that I wrote the following little Bash script and run against the JBOD array from another computer. The script creates numbered files, each with a size of 1MB. 1000 such files are placed in each directory. The plan was to fill the entire disk and then take out the disks to study and see what the DNS-323 had stored on each disk and to verify that the content on a disk would in fact be accessible if the other disk broke down.
#!/bin/bash
TARGET=/mnt/nas
FOLDERS=500
FILES_PER_FOLDER=1000
BLOCKS_PER_FILE=2000
if [ ! -d $TARGET ]; then
echo "Target folder does not exist"; exit
fi
for d in `seq 1 $FOLDERS`; do
dirname=`printf 'D%07d' $d`
echo "Creating folder: $dirname"
mkdir $TARGET/$dirname
echo " Creating file: F0000001"
dd if=/dev/zero of=$TARGET/$dirname/F0000001 count=$BLOCKS_PER_FILE
for f in `seq 2 $FILES_PER_FOLDER`; do
filename=`printf 'F%07d' $f`
echo " Copying file: $filename"
cp $TARGET/$dirname/F0000001 $TARGET/$dirname/$filename
done
done
Please check back for the result of these tests.
I have had my Dlink NAS DNS-323 since early 2007. It has mostly served me well. Over the months I have put more and more files on it so that it now holds about 350GB of data. Out of fear of losing precious data I have not updated the firmware so I am still on 1.03 from May 2007.
I mounted a shared folder on the DNS-323 from a Ubuntu client and noticed that the Swedish characters were all messed up. First I thought the error was related to how I mounted the drive from Linux, but then I found out that the issue is with the DNS-323 itself and the fact that it uses a non-Unicode character set for the filenames. This should be solvable with the iocharset and the codepage parameters to the mount command in Linux but I couldn’t get it to work.
Later firmwares are said to fix the problem – but only if the drives are totally wiped. I got myself a USB drive sufficiently large to hold everything and copied all the data over using rsync so now I am just about ready to upgrade the firmware and reformat the drives and use some of the plugins on http://wiki.dns323.info. But more on that some other time.
Before I wipe the disks I wanted to make sure that I could rename all the files using Unicode but with some 50,000 files I didn’t want to do it manually. The Linux command iconv can convert between encodings but it works on a file level and I wanted something that only touches the filenames, not the contents of the files.
I found the Perl command convmv which is available through the standard Ubuntu repositories. Just type “apt-get install convmv”. It does the same as iconv but on filename level. Precisely what I needed. I then typed:
#/mnt/wd640gb# convmv -f cp850 -t utf8 -r .
This command shows how files would be renamed, switching from codepage CP850 (the default or DNS-323) to UTF8. Once you are happy with the suggestions, just issue the command again but with the extra switch –notest to actually rename the files.
My only issue now is that convmv only works on filenames, not directories. But at least I have reduced by problem by a factor 30 or something. The directories I can do manually.
I have been thinking about doing some development for iPhone and have come up with some really cool application ideas that I would like to turn into real applications. However, after reading on various blogs I have realised that the limitations in the development environment on the iPhone mean that those applications are not possible; at least not in a way that would allow me to distribute them through AppStore. That, of course, means that they are dead in the water since the vast majority of iPhone users will only use the AppStore.
What is holding me back? Two words – background processes.
The sad story is that third-party applications can not be made to stay running in the background when the user switches task, takes a call or when the phone goes into sleep mode. Of course, this limitation does not apply to Apple’s own applications. The argument that Apple is pushing is that they need to ensure that background processes don’t slow down the phone or drain the battery. This may seem valid but personally I think it should be left for the end user to decide. It could even be done so that the AppStore clearly states how long the application may be running in the background before it is installed – after which time the OS shuts it down.
I hope this is fully addressed by Apple soon – and no, the promised notification service is nowhere near solving my problem. Otherwise I may go back to developing my applications for Symbian or Windows Mobile – both of which I have written applications for in the past.
I have set up a number of server based on the past four or five versions of Ubuntu Server. Every time I face the dialog where one can pick what additional software to install, I just select OpenSSH so that I can make the server headless and manage it remotely. I then install each individual package using apt-get until the system works as I wish.
First, I thought that the alternatives for software could be installed as meta packages using apt-get but I never managed to find any such meta packages in the package list on http://packages.ubuntu.com – or by searching the repositories with apt-cache.
I just now learned that the command tasksel brings up a menu similar (but extended) to the one shown during the installation. It is also possible to list which deb packages an individual task would install. A simple method would for instance be to run ‘tasksel install dns-server’ which would lead to the exact same result as if one had picked the DNS server from the beginning.
