Trying to evolve

We Finns have proverb about learning by "kantapään kautta", basically that means that we learn from our mistakes. This must be the one of most painful methods of education, but it sure is effective. If you are wondering what I am babbling about, then please read my first blog entry, because this entry is about BACKUPS.


An easy and pretty fast way to make a backup of Gentoo installation is by using script called Stage4. This archives your root partition either into single or multiple gzip's or bz2's. The script is easy to use and does not need much configuration. I am currently running it with default settings. During my first backup my root partition used 2.7GB and was compressed to 552MB. I am storing the backup-archive on my workstation, because even I cant wipe out two computers at once.

You can download the stage4-script from here. I advice you to read through it, the configuration is in the beginning, it is very well documented and is not that long. But basically you just drop the file to /usr/local/bin, chmod it to have execute rights and then execute the script. Script reports problems if there are some and the output is pretty self explanatory. Hopefully you don't have to learn the restoration process, but if you do, it is documented in the wiki.

Once my installation is ready, or at least "more complete", I will be writing a script that makes stage4 archive every week or so, and then transfers it to ftp-server. I also am planning on cloning the whole partition with a tool like dd or partimage, latter being a better (and more complicated) solution, because it does not backup the empty space on a partition. Current situation with my mirrored root drive gives some protection against a hardware malfunction, but offers only little against user mistakes.

So for the time being I am making stage4 archives from my root partition as backups and if you are using Gentoo, I advice you to do at least the same.

Installing HTPC software (part 1)

I'll start this one of with a quick and easy way to get samba file sharing working. If there are no interest in having such service, then you can skip over the Samba part. After the Samba installation I will install software that is needed for the HTPC functionality.

But first, here is a make.conf file that I am using. You may want to edit it to better suit your needs. It has basicly USE-flags enabled that "sound" like they might be useful in HTPC installation.


Samba
Before emerging samba, there is again a file in need of an extra line. This time easiest way to add it, is by running following command

echo "net-fs/samba acl -cups -doc -kerberos -ldap -oav pam -python -quotas readline -selinux winbind" >> /etc/portage/package.use

This will ensure samba will be emerged for just basic file sharing, with nothing too fancy like cups or ldap. Modify as needed. After the above line is added, just emerge samba. After all that is done, it is time to edit /etc/samba/smb.conf to suit your file sharing needs.

Here is a little example of how my smb.conf looks like

[netlogon]
comment = Network Logon Service
path = /var/lib/samba/netlogon
write list = root

[global]
add machine script = /usr/sbin/useradd -c Machine -d /var/lib/nobody -s /bin/false %m$
domain logons = Yes
domain master = Yes
local master = Yes
os level = 65
preferred master = Yes
security = user
workgroup = Workgroup Name
map to guest = Bad User
include = /etc/samba/dhcp.conf
logon path = \\%L\profiles\.msprofile
logon home = \\%L\%U\.9xprofile
logon drive = P:
netbios name = Server name
load printers = no
printing = bsd
printcap name = /dev/null
disable spoolss = yes

[profiles]
comment = Network Profiles Service
path = %H
read only = No
store dos attributes = Yes
create mask = 0600
directory mask = 0700

[yoursharehere]
comment = Yoursharehere is a shared directory
inherit acls = No
path = /path2yoursharehere
read only = No
browseable = Yes
writable = Yes
force user = yourusername


After the smb.conf is set up correctly, only thing left is to add samba users and their passwords. I use the following command to create first the user, then adding samba password to that user. Note that the username password should be preferable the same ones you use in your clients. That way a windows user can mount the share as a network drive by just using his own username and password. After adding the users, you are ready to start the samba service and add it to some runlevel, so it will start automatically on boot.

useradd yourusername && smbpasswd -a yourusername


Enabling the Gentoo Portage Multimedia Overlay
Multimedia overlay is used to get the latest (and greatest) svn/cvs -versions of the multimedia programs like mplayer, which is used as the default video player in MMS. The reason we need the latest version is that it has multi-thread support enabled. This helps a lot when playing content that is encoded with H.264 with high definition resolutions. Playing 1080p material with h.264 encoding used to be pretty impossible with modest 3ghz dual core like mine. Nowadays with multithreading, it handles it pretty nicely.

This is how the overlay is enabled

mkdir /usr/local/portage
cd /usr/local/portage
rsync -rptvW rsync://kpex.no-ip.org/kpex-media
echo "PORTDIR_OVERLAY=\"/usr/local/portage\"" >> /etc/make.conf


HTPC software
Xineliboutput is a plugin to VDR, it is used to output the DVB-material via graphics card, either in X (vdr-sxfe) or in frame buffer (vdr-fbfe). Normal use case with VDR is that user has full feature DVB-tuner which has some kind of TV-out, but with high resolution flat panels this is hardly optimal. Instead we are going to use the DVI output on the graphics card to transfer the picture in HD resolution to TV´s HDMI input. Xineliboutput enables us to do just that. The plugin itself is loaded when VDR starts and the client (vdr-sxfe) is started whenever we select "Watch TV" from MMS main menu. And finally to the point: Xineliboutput requires xine-lib to be installed.


Besides xine library and mplayer we also need some codecs. W32codecs contains Microsofts decoders for codecs used in wmv-container (for example). We also should install x264-svn package. And lastly you can install MMS right here, without any extra steps, IF you are content (and smart) to use the stable version. I however, am going to go the extra mile and try out the new release candidate 1.1.0 version.

All of the above are installed by simply (mmsv2 installs at the time of this writing the stable 1.0.8.5 version of the MMS) emerging the following

emerge xine-lib mplayer win32codecs x264-svn mmsv2


Disclaimer: While the Xineliboutput enables showing the DVB material in high resolution X environment, it does not actually do anything to make the picture better. You are still faced with the limitations of the source material, which is often low resolution and encoded with low bitrate. The upside is though that the DVB viewing with VDR can now be integrated as part of the X (the graphical interface) and more importantly part of the MMS.

The Masterplan™

Now that all my hardware seems to be working correctly, it is time to move to installing needed software. But before that I guess I should let you in on my plan for the final HTPC.

I am going to use the My Media System (MMS in short) as a Media Center software. I had the stable version 1.0.8.5 installed on my previous installation. I liked it very much, it seems to be stable, feature rich and simple at the same time. Also the new version 1.1.0, which is currently in release candidate -state, seems to have many nice new features. I think I am at least going to try the new bleeding edge version first and if it is still too unstable, change back to the 1.0.8.5 version.
While MMS takes care of the things like picture viewing, music and video playing and providing nice user interface for these, I am going to use VDR for the DVB side. I've been using VDR to watch DVB-television for some time now (over a year at least) and I am very happy with it.
Other option would have been MythTv which bundles all of this together and provides even more functionality. But MythTv has been more viable on the analog TV side, not so good in DVB. So basically VDR with X frontend does the watching and recording of television and MMS does the rest.

Here is a little demonstration of how both VDR and MMS work together. Keep in mind that the video is old and for example exiting the VDR frontend (the AVP movie) is more seamless nowadays.

Beyond of being "just" a Media Center for my living room, I am also going to utilize it as a file server. My home network includes two workstations and one laptop and because of the WLAN is available, I also have some visitors when friends come over with their WLAN enabled devices, like laptops or mobiles.

I will be using Samba for file sharing between the OpenSuse workstations, laptop and mostly Windows-based visitors. Samba-server is pretty easy to setup and making connections by clients is also very easy. Mirrored hard drives (RAID1) on my HTPC slash file server will take care of the content I don't want to lose due to hard drive crash. I will also probably make a little script that makes backups from HTPC/File server to the net.


OK, so maybe in my next entry I will be actually getting to the software installation phase, which I already promised few entries back.

Getting the hardware to work

Fortunately one thing I was able to rescue from earlier, corrupted, installation is the kernel config. So I don't need to spend much time with kernel configuration. I however decided to go bleeding edge with the just released 2.6.24 kernel. Preserving the old configuration is pretty easy, just copy the old .config-file to the new kernel sources (/usr/src/linux) and run "make oldconfig" the command asks user to answer few questions and voilá, kernel configured.

Installing DVB drivers is also pretty easy and there are few different ways to do it. I used to install them just by using emerge, ie. "emerge mercurial && emerge v4l-dvb-hg" but currently there seems to be compilation problem. Because of the problem with emerge I had to install the drivers by hand. Here is a short overview of the process, the guide is for different tuner card than mine, but one can see the outline of the process easily enough. After dropping the firmware file to correct location (/lib/firmware) and rebooting, udev should load the modules for the card automatically. One can verify that the tuner is working by making sure there is device called /dev/dvb/adapter0 .

Lirc installation is also pretty straightforward, although it can also be problematic for the first timers. For emerge to work correctly a line needs to be added to /etc/make.conf, with my Silverstone (iMon) IR-receiver the line looks like this
LIRC_DEVICES="lirc_imon imon_pad2keys imon imon_pad imon_rsc"

After that "emerge lirc" and reboot the /dev/lirc device should be available. With serial port IR-receiver some tweaking with "setserial" command might be required. Correct lircd.conf is of course needed in /etc directory. For most remotes you don't need to look any further than here. I however made my own for the Harmony remote by using "irrecord" command. Testing that lirc is working command "irw" can be run, while running it should detect key presses from the remote.

If you want to use the remote that came with the case (the one with a "mouse pad" on it) you should have look at this page.

Once the lirc is working, emergin lcdprog makes the VFD available for using. Testing that it is working is easy, just "echo whatever > /dev/lcd0".


So what else is there?

• Proprietary Nvidia graphics drivers can be installed by issuing "emerge x11-drivers/nvidia-drivers" command and then using "modprobe nvidia" to load the module, although I think udev also loads automatically after reboot. Remember to include the nvidia as a driver to your /etc/X11/xorg.conf if you are using X.
• My integrated sound card is recognized automatically and my sound is working after "emerge alsa-utils" and setting the volume with "alsamixer", using "alsactl" one can store the volume settings as defaults.
• Integrated network was correctly detected during install and if it is not and you can not get it to work by hand, installing Gentoo becomes pretty hard. Luckily ethernet works correctly at full gigabit speed.
• Flat panel TV can sometimes be a real pain to get working correctly due to the TV itself giving wrong information about its capabilities. Example xorg.conf file for Mirai DTL-642E500 can be found here. I personally had some problems with finding correct settings and hopefully the linked file is useful to others.

Disclaimer: To get the devices to work, the kernel needs to be configured correctly. But that information is (mainly) available at Gentoo wiki, so there is no need for me to rewrite it all here. Check the Howto Index, Tips & Tricks and Hardware part of the wiki.

Basic Gentoo Installation

As I mentioned before, even the basic Gentoo installation can be a chore. But after installing Gentoo several times, I am starting to see the beauty behind it. Everything is done on command line but in a sense, I have full control of how my system is going to be. Nothing unnecessary gets installed or setup, which is great help when you´re trying to minimize the amount of software.
For example boot process slows considerably by adding more services,
so one should be aware what is actually needed and what can be left out. Also less software means less disk usage, less security issues and most importantly more performance.

Installation process is pretty easy if one follows the Gentoo Handbook and does not deviate much. I decided install to RAID1 array right away to steer clear of the pitfalls of doing it later.
Installation itself was pretty uneventful, but I would not recommend it to any one without proper Linux experience.

There are easier distros to install, or rather, almost all other distros are easier to install. One could go with Ubuntus minimal server installation and make a pretty decent HTPC. Not many however offer you so much control over your OS as Gentoo.


I decided to keep the partiotioning of the disks as simple as possible.

• /dev/md0 10GB for / (root) partition
• /dev/md1 290GB for /safe partition, this is used to store important data
• /dev/md2 4GB for swap partition, yes I know that is pretty much, but I want to be able to Suspend my RAM to disk if need arises.

So far I've emerged (installed) the following additional software.

• syslog-ng and logrotate
• vixie-cron
• grub
• mdadm
• xfsprogs (this is needed for my 500GB DVB Recording XFS partition)
• alsautils
  

And I have enabled following additional services.

• sshd
• syslog
• eth0
• mdadm
• syslog-ng
• vixie-cron
• alsasound
  

I am planning to start the not-critical services in their own runlevel or at least after the HTPC services that are needed to make the Media Center start as fast as possible. Currently the mentioned services still reside on default-runlevel.

Next time, installing Lirc remote control software, Mplayer video player, X the graphical server and Samba for network file sharing and maybe few others.

Hardware Specifications

Let´s start this one of with hardware specifications (everyone loves those, right?) of the HTPC. This should give some idea of what I am trying to accomplish and maybe some pointers of what I have considered to be good hardware for such project. Let´s face it, it is also the easiest place for me to start.

Hardware specifications:

• Processor: 2.0ghz Intel Pentium Dual-Core E2180
  
• Motherboard: Gigabyte GA-P35-DS3R (rev. 1.o)
  
• Memory: 2x 1GB Transcend 800mhz aXeRam
  
• Hard disks: 2x 320GB and 500GB Seagate Barracuda SATA drives
  
• Graphics: Nvidia Geforce 6800 GS
  
• DVB tuners: 2x Technotrend Premium C-2300 Hybrid (aka Hauppauge Nexus-CA ?)
  
• PSU: Seasonic S12II-430W
  
• Enclosure: Silverstone LC20M with IR and VFD display
  
• Cooling: Scythe Mini Ninja CPU cooler, 2x 92mm and 2x80mm silent fans
  
• Accessories: Logitech Harmony 555 remote and Ione Scorpius P20 Media Center Keyboard
  

Here is a little rationale about why I have chosen the hardware above.

First of all I decided to build my HTPC setup around Intel E2180 processor, not just because it is dirt cheap (72€ here) but also because every E21x0 processors overclocks like nothing else! I am talking 100% overclock with stock cooler and with a little bump to core voltage. I am currently running the 2.0ghz version at 3.0 ghz and it has been rock solid for few months now. It also handles H.264 encoded FullHD video easily (albeit the software, mainly mplayer, needs to be CVS version that includes multithreading support).

The Gigabyte motherboard seems to be good choice, it is very stable even with overclocked system like mine. It also has 3 older PCI slots which are useful because I have 2 PCI slot DVB tuners and I might consider adding third one. The 1.0 revision of the board also has 4 additional USB headers which are very useful because the enclosure alone needs one for the VFD and one for the USB/Audio front panel, I have also thought of adding memory card reader in the front panel, which would require one more USB header. Being Gigabyte the motherboard also features their "Ultra Durable 2" design, which seems kinda cool.

Transcend memory was chosen because of it's overclocking potential and also because it was quite cheap at the time. I managed to get hold of a batch of memory that still included the Micron memory chips which are know for their uncanny overclocking potential. Although I really do not have so much use for the "potential" since I am running the memory near the stock settings. Overclocking the CPU was done by increasing the FSB and at the same time lowering the memory multiplier.

With three hard drives and three cards in slots the case is getting a bit crowded, but I thing I will manage to add one more drive. The two 320GB drives are going to be used in RAID1 (mirroring) setup and will store the Gentoo system (root and swap) partitions and also serve as "safe" place to stores data I don't want to lose, like my pictures and music. The 500GB drive currently houses DVB recordings and will be fully dedicated for this purpose. The fourth drive that I am going to add at some point is going to be dedicated to house all
the multimedia content, mainly high definition video.

Nvidia Geforce 6800GS is likely a bit overkill for the use in HTPC, but that is what I had available. Nvidia does not provide much of support for hardware based video decoding in Linux, which causes the need for such computing power as E2180@3ghz. For example Nvidia 8400GS would be ideal for HTPC because it supports hardware decoding of H.264 video and also has HDMI output.
Unfortunately Nvidia has decided to leave Linux users hanging.

Technotrend DVB tuners are so called full feature cards, which are leftovers from my original VDR-box (Video Disk Recorder). Each card can display and record one full DVB-mux at time, so currently I am able to view and record from two different program muxes at the same time. There are some times that I would like to have capability to record from two different muxes and at the same time watch third. Fortunately there is still room for expansion.

Seasonic PSU was chosen because of it's very good efficiency, low noise and good quality. First I was going to buy the 380 wattage version, but availability was a problem, so I went with the bigger one.

Tastes differ, but the looks is always a big priority when choosing casing for a HTPC. I've always liked the looks of the Silverstone cases, in particular I liked the LC20M which hides optical drives and front IO ports behind doors. The case does not stand out at all amongst other entertainment electronics I have under the TV-set. Also I liked that the case comes with a built in IR-receiver and VFD.

Cooling can be a problem in HTPC cases, which are built for looks and compactness, not for optimal cooling results. LC20M has mounts for 2 92mm fans in front of the HD-cage and for 3 80mm fans near the CPU cooler. Adequate cooling can be provided by using slow turning fans making minimal noise. Currently I still have the stock Intel cooler but I am going to replace it within days with a Scythe Mini Ninja cooler. Cooling fans are currently Nexus made, but once I get rid of the stock sink I'll have to review if they are silent enough.

I'll get back onto Logitech Remote and Iolo keyboard later.

Starting a new blog

Few nights ago I managed to corrupt my root-partition on my Linux-based HTPC machine. I was a little tired when migrating the system to RAID1 configuration, didn't make complete backups, and of course managed to hose things pretty badly. Ironically my objective was to make my system more durable by mirroring the system disk to second hard drive. Long story short, reiser filesystem in my / (root) partition is corrupted and no amount of reiserfsck-magic can make it whole again. So now I need to re-install the whole Gentoo based operating system.

At least that serves as a good learning experience, I am going to be more diligent about making backups from now on. This blog serves as a method to "back up" information about my installation process, so I have someplace to refer to when I am installing new HTPC system and can not exactly remember what needs to be done. Gentoo installation itself can be quite complicated process (at least when compared to easy-to-use modern distros like OpenSuse and Ubuntu) and making it fully HTPC cabable is not easy either.

And who knows, maybe this blog can help someone facing same problems.