Linux

Notes on various Linux related items.

Python:


Networking

General Linux Stuff

HTPC Stuff:

HTPC Stuff:

  • Using OSM data for routing

    Openstreetmaps contains a lot of detailed infvormation of many parts of the world; This information is contained in an OSM-file, and can be freely used for any application.

    My goal is to perform route-calculations using Open Street Map data (i.e. determine possible routes from A to B and find the shortest path).
    There is quite some information available on the web, however some of it seems outdated a bit. ( outdated to the extend that some of the steps to take or commands to use are no longer exactly valid, although the general idea is still the same).

    These are my notes on how to get a working system that allows me to use OSM-data for determining a route, using Spatialite, an SQLITE-version adapter for use with geoinformation and spatial queries..

    Preparation steps/ installing packages
    Prerequisits, or what to do first:
    we need to install spatialite and spatialite tools.
    On Gentoo:
    emerge -av spatialite spatialite-tools
    Spatialite is now available on your system, and running “spatialite” give a similar database-terminal as when running ‘sqlite’

    Overview: Start routing.
    Since the actual routing-algorithm is performed by operations in the database some preparation is needed for the data to be ready for use in routing:
    - Get an OSM-file for an area of interest ( i.e. your hometown :-) )
    - Use ‘spatilite-tools to import the data in an sqlite/spatialite database. ( spatialite is sqlite with some additional functionalite specific for routing purposes).
    - Create the necessary routing info (again using spatialite-tools).
    - Start querying the spatialite-database, either directly ( command prompt), or using pyspatialite , a modified version of pysqlite that takes into account the extra functionality of a spatialite-database (compared to an sqlite database).

    Get an OSM-file
    - Go to Openstreetmap,
    - Select the “export”-tab ,
    - Select (either manually or by using lon/lat coordinates) an area
    - Export.

    You are only allowed to export a small area due to server limitations, but larger files can be downloaded completely at various locatios, for an overview see the download-section.
    However, when starting to use Spatialite, better start with a small area for testing purposes ( since processing whole continents takes some more processor-time).

    Import in Spatialite / sqlite database
    This step assumes you have an OSM-file called name.osm (replace name with the actual name).

    Create an sqlite-database from the osm-file:
    spatialite_osm_net -o name.osm -d .sqlite -T roads -m

    Next, create a new table containing network information in the database (note: next command is one command, and should be placed on one line in bash):
    spatialite_network -d name.sqlite -T roads -g geometry -c length -t node_to -f node_from -n name -o roads_net_data

    Last, create a table called “roads_net_tabel” with “graph”-information about the roads connection various nodes in your selected area:
    spatialite name.sqlite 'CREATE VIRTUAL TABLE "roads_net" USING VirtualNetwork("roads_net_data")'
    (again, this is one line, not two as it might appear in your browser)

    See this link about routing using spatialite, specifically the paragraph “Creating routing data”.

    Start Routing
    Start spatialite using
    spatialite name.sqlite
    Now you entered a regular sqlite-shell.

    Start routing using:
    select * from roads_net where nodefrom = 1840 AND nodeto = 2638 ;

    This simple command now queries the database for a route from node 1840 to node 2638. ( Note that the nodes I query from and to are chosen randomly by me in this example).
    Result should look something like the following:
    Algorithm|ArcRowid|NodeFrom|NodeTo|Cost|Geometry|Name
    Dijkstra||1840|2638|160.67405722996||
    Dijkstra|2377|1840|1874|13.1715757332811||
    Dijkstra|2393|1874|1865|2.71515965136783||
    Dijkstra|2400|1865|1883|2.24290239479153||
    Dijkstra|1577|1883|1916|3.68138351828062||
    Dijkstra|4646|1916|2194|45.9721562356986||
    Dijkstra|1981|2194|2537|55.9335573133834||
    Dijkstra|1977|2537|2637|35.9918700870997||
    Dijkstra|1975|2637|2638|0.96545229605724||

    If you are unsure if the nodes 1840 or 2680 exist ( fe because you selected a small OSM-area with only a few tens of nodes), test the routing using:
    select * from roads_net where nodefrom = 1 AND nodeto = 3 ;

    if somehow this doesnt return any results, ther emight have been a problem when creating the routing-info.
    Try ( in the spatialite-shell):
    spatialite> select * from roads_nodes limit 10;

    This should give something like the following, where the first column is the node-ID in this database, and the second column is the OSM-node-id:
    1|43791847|1|
    2|43800255|1|
    3|43735986|1|
    4|1493825604|2|
    5|1419273601|1|
    6|43792337|3|
    7|43768546|3|
    8|1419273597|3|
    9|43791469|3|
    10|43803689|1|

  • libtiff.so.3: cannot open shared object file: No such file or directory

    A common error, especially when mixing stable (amd64) with some unstable (~amd64) packages, is that versions needed and versions present of some exotic application do not align, and sometime cannot allign since portage can only install one.
    In my case, I ran into this error using PlayOnLinux:

    lex ~ $ playonlinux
    Traceback (most recent call last):
    File "mainwindow.py", line 30, in
    import wx
    File "/usr/lib64/python2.7/site-packages/wx-2.8-gtk2-unicode/wx/__init__.py", line 45, in
    from wx._core import *
    File "/usr/lib64/python2.7/site-packages/wx-2.8-gtk2-unicode/wx/_core.py", line 4, in
    import _core_
    ImportError: libtiff.so.3: cannot open shared object file: No such file or directory
    ./playonlinux: line 96: python2.6: command not found
    lex ~ $

    So, the problem:
    libtiff.so.3 is missing !

    First, check what version is there, and what version(s) is/are not:

    cd /usr/lib
    ls -l |grep libtiff

    This give the available version of libtiff, in my case:

    lex /usr/lib $ ls -l |grep libtiff
    lrwxrwxrwx 1 root root 16 Jan 28 19:32 libtiff.so -> libtiff.so.5.0.5
    lrwxrwxrwx 1 root root 16 Jan 28 19:32 libtiff.so.5 -> libtiff.so.5.0.5
    -rwxr-xr-x 1 root root 472112 Jan 28 19:32 libtiff.so.5.0.5
    lrwxrwxrwx 1 root root 18 Jan 28 19:32 libtiffxx.so -> libtiffxx.so.5.0.5
    lrwxrwxrwx 1 root root 18 Jan 28 19:32 libtiffxx.so.5 -> libtiffxx.so.5.0.5
    -rwxr-xr-x 1 root root 10136 Jan 28 19:32 libtiffxx.so.5.0.5

    Ok, become root and create a link called ” libtiff.so.3″ to “libtiff.so.5.0.5″.
    lex lib # ln -s libtiff.so.5.0.5 libtiff.so.3

    In this whay, the program can find the right libtiff, but it actually is another one.
    Note that this *might* or *mightnot* work, depending on the exact changes inversions between the one needed and the one present. Sometimes, it just works, problems solved.

  • WSGI

    WSGI (Web Server Gateway Interface) is designed to allow communication between Python and web-applications. You can build webapplications in Python that you can call from a website, or WSGI can be used to create an entire web-framework.

    WSGI is replacing Mod_Python as a way to use Python in interactive web programming. THis page contains some simple example-scripts to start using WSGI.

    Prerequisite
    You first need to install WSGI on your server; On Debian this can be installed using apt-get install libapache2-mod-wsgi

    Getting started: Small Python/WSGI programs

    ‘Hello world’ using WSGI

    filename: helloworld.wsgi
    def application(environ, start_response):
    status = '200 OK'
    output = ' Hello World!
    '

    response_headers = [('Content-type', 'text/html'),
    ('Content-Length',str(len(output)))]
    start_response(status, response_headers)
    return [output]

    So, this looks easy: Create a file called helloworld.wsgi and place it in the wsgi-app directory (per default on Debian this is /usr/local/www/wsgi-scripts ).
    When pointing your browser to the right location + filename, the function ‘application’ is executed.
    Some strings are defined (status, output), the start_response command outputs both content-type and content-length, and finally the output is returned as if it where a ‘real’ html file.

    ‘Hello world Extended’ using WSGI
    Now, to get some information from the webserver to the application we extend the program using wsgiref.util.

    filename: helloworld2.wsgi
    import wsgiref.util
    def application(environ, start_response):
    status = '200 OK'
    output1 = ' Hello World!
    '
    output2 = 'requested URL = '+ wsgiref.util.request_uri(environ, include_query=0)
    output = output1 + output2
    response_headers = [('Content-type', 'text/html'),
    ('Content-Length',str(len(output)))]

    start_response(status, response_headers)
    return [output]

    Show IP of webpage visitor using WSGI

    import wsgiref.util
    #from paste.request import parse_formvars

    def application(environ, start_response):
    status = '200 OK'
    headers = [('Content-type', 'text/plain')]

    start_response(status, headers)

    IP_adres =environ.get('REMOTE_ADDR')

    start_response('200 OK', [('content-type', 'text/html')])

    Header = ''' Your IP is:

    '''

    End_tekst '''

    To see your IP v6 address, connect to this page using itsIPv6 address

    '''

    #return [environ.get('REMOTE_ADDR')]
    return [Header + IP_adres+End_tekst]

  • Python Image manipulation

    Python can be used to manipulate or create pictures. One way is to use PIL , the Python Image Library. This page contains some simple examples to get started.

    For the examples below this image will be used for testing the apps. Download the image and save it as “Linux_Logo.png” in the same directory as the python scripts ( or change the filename in the script).

    Example 1: Read values .png file per pixel
    This program determines the size of the picture, and then reads and prints the values of each pixel:


    from PIL import Image
    png_file_name='Linux_Logo.png'
    # change to the filename. use full path if the png-file is not in the same dir.

    def MAIN():
    im = Image.open(png_file_name)
    pix = im.load()
    image_size= im.size
    print 'image size is ' , image_size
    xrange = image_size[0] # set the x-range of the image from the image size.
    yrange = image_size[1]

    for j in range (0,xrange):
    for i in range(0,yrange):
    value = pix[j,i]
    print value

    MAIN()

    Example 2: Read values and change first component (red in RGB) to 255
    This program results in the logo to be transformed to a more ‘red-coloured’ picture.
    It reads all pixels, changes the value of the first byte in RGB ( RED) to 255, and keeps the other colours.
    The result should be something like this:


    from PIL import Image
    png_file_name='Linux_Logo.png'
    # change the filename to the picture you want to use.

    newData = list()

    def MAIN():
    img = Image.open(png_file_name)
    pix = img.load()
    image_size= img.size
    print 'image size is ' , image_size
    yrange = image_size[0]
    xrange = image_size[1]

    for j in range (0,xrange):
    for i in range(0,yrange):
    value = pix[i,j]
    new_value= (255,value[1] , value[2],value[3])
    #Example:
    #Make first (Red in RGB) byte 255 ( maxvalue)
    #keep other components what they were.
    newData.append(new_value)

    img.putdata(newData)
    img.save( "result.png")

    MAIN()

  • MyIP

    What is my IP address?
    To see you IP address ( v4 or v6) click down here:

    Show IP

    What is my IPv6 address?
    The above link *should* work using either IPv6 or IPv4 addresses. However, if you see an IPv4 address, and where expecting an IPv6 address, you can try activating the script using the IPv6 address of this page.

    Show IP Using IPv6 address
    [2001:4b98:dc0:41:216:3eff:fea8:e73d]

    Q: Why don’t I see an IPv6 address ?
    A: Depending on how you enter the IPv6 web, you might see either an IPv6 or an IP v4 address. If you are part of the IPv6 internet you should see your IPv6 address. However, if you use IPv6 via an IPv4-2-IPv6 adapter, or by ‘tunneling’ IPv6 traffic over an IPv4 network to the IPv6 internet via some IPv6 ready proxy, DNS resolving might or might not work, or you might see the address of your proxy

    Q: Why do I see an IPv6 address using the direct link ? I am on an IPv4 computer/network…
    A: Many computer systems have build-in ‘virtual adapters’ that allow browsers to navigate to IPv6 address using some remote IPv6 proxy. However, DNS resolving and using Domainnames might not work.

Networking

  • Network Tools

    Show network Trafic / bandwidth usage

    There are various tools to get an idea of the current bandwidth usage oon your system for the various network adpaters you might have.
    A simple tool is Iftop; You can also use IPTRAF.
    A short introduction to both is given down here:

    IFTOP
    ‘iftop’ is a fairly simple tool that analyses network traffic on your network interface and gives bandwidths for up and downlink traffic.
    Install by using (gentoo) :
    # emerge --ask -v iftop
    This should give something like:
    MediaTestServer ~ # emerge --ask -v iftop
    These are the packages that would be merged, in order:
    Calculating dependencies... done!
    [ebuild N ] net-libs/libpcap-1.1.1-r1 USE="-bluetooth -ipv6 -libnl -static-libs" 569 kB
    [ebuild N ] net-analyzer/iftop-0.17 157 kB
    Total: 2 packages (2 new), Size of downloads: 725 kB
    Would you like to merge these packages? [Yes/No] y

    Yes, of course :-)

    Now, to get an idea of the trafic on your system, run iftop:
    # iftop
    interface: eth0
    IP address is: 192.168.0.30
    MAC address is: 00:1a:92:03:e0:a1

    IPTRAF
    IPTraf (describing itself as a “Interactive Colorful IP LAN Monitor” indeed has some more colors ( allthough not that much more) and is a commandline interactive frontend for <> that gives various forms of network information including bandwidths and TCP and UDP info.
    Install (emerge) using:
    # emerge --ask -v iptraf-ng

    Giving:

    MediaTestServer ~ # emerge --ask -v iptraf-ng
    These are the packages that would be merged, in order:
    Calculating dependencies... done!
    [ebuild N ] net-analyzer/iptraf-ng-1.0.2 USE="-static-libs" 857 kB
    Total: 1 package (1 new), Size of downloads: 857 kB
    Would you like to merge these packages? [Yes/No] y
    >>> Verifying ebuild manifests
    >>> Emerging (1 of 1) net-analyzer/iptraf-ng-1.0.2
    >>> Downloading 'http://distfiles.gentoo.org/distfiles/iptraf-ng-1.0.2.tar.gz'
    --2012-01-22 17:25:26-- http://distfiles.gentoo.org/distf

    And just start using the following command give a lot of real-time infomration about network usage, allowing debugging or determining the load caused by specific system processes:

    MediaTestServer ~ # iptraf

  • System specs

    Hdparm: Show Disk read write performance

    One can use hdparm to get an idea for harddisk or ssd reads using:

    # hdparm -t /dev/sda
    Typical output can be:
    /dev/sda:
    Timing buffered disk reads: 328 MB in 3.02 seconds = 108.74 MB/sec

    This gives a starting point when benchmarking your system, and determining bottlenecks in performance. Actual read and write times might be different, since hdparm reads a continuous part of the harddisk / solid state disk , and ‘real’ usage usually consists of many random reads of different parts of the disk

    Dmicode: Show memory type and settingsd
    To show RAM-type (DDR,DDR2, DDR3 etc) and actual bus speed use:
    #dmidecode --type 17
    Output:
    # dmidecode 2.9
    SMBIOS 2.4 present.

    Handle 0x002C, DMI type 17, 27 bytes
    Memory Device
    Array Handle: 0x002A
    Error Information Handle: Not Provided
    Total Width: 64 bits
    Data Width: 72 bits
    Size: 1024 MB
    Form Factor: DIMM
    Set: None
    Locator: DIMM0
    Bank Locator: BANK0
    Type: DDR2
    Type Detail: Synchronous
    Speed: 667 MHz (1.5 ns)
    Manufacturer: Manufacturer0
    Serial Number: SerNum0
    Asset Tag: AssetTagNum0
    Part Number: PartNum0

    Handle 0x002E, DMI type 17, 27 bytes
    Memory Device
    Array Handle: 0x002A
    Error Information Handle: Not Provided
    Total Width: 64 bits
    Data Width: 72 bits
    Size: 1024 MB
    Form Factor: DIMM
    Set: None
    Locator: DIMM1
    Bank Locator: BANK1
    Type: DDR2
    Type Detail: Synchronous
    Speed: 667 MHz (1.5 ns)
    Manufacturer: Manufacturer1
    Serial Number: SerNum1
    Asset Tag: AssetTagNum1
    Part Number: PartNum1

    Show actual Ethernet Linkmode
    As root run
    ethtool eth0

    To get detailed information about the supported link modes, and the actual used speedmode of your ethernet connection.
    Typical output:

    root@mediasystem:/home/lex# ethtool eth0
    Settings for eth0:
    Supported ports: [ MII ]
    Supported link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Full
    Supports auto-negotiation: Yes
    Advertised link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Full
    Advertised pause frame use: No
    Advertised auto-negotiation: Yes
    Speed: 1000Mb/s
    Duplex: Full
    Port: MII
    PHYAD: 3
    Transceiver: external
    Auto-negotiation: on
    Supports Wake-on: g
    Wake-on: d
    Link detected: yes

  • MyIP

    What is my IP address?
    To see you IP address ( v4 or v6) click down here:

    Show IP

    What is my IPv6 address?
    The above link *should* work using either IPv6 or IPv4 addresses. However, if you see an IPv4 address, and where expecting an IPv6 address, you can try activating the script using the IPv6 address of this page.

    Show IP Using IPv6 address
    [2001:4b98:dc0:41:216:3eff:fea8:e73d]

    Q: Why don’t I see an IPv6 address ?
    A: Depending on how you enter the IPv6 web, you might see either an IPv6 or an IP v4 address. If you are part of the IPv6 internet you should see your IPv6 address. However, if you use IPv6 via an IPv4-2-IPv6 adapter, or by ‘tunneling’ IPv6 traffic over an IPv4 network to the IPv6 internet via some IPv6 ready proxy, DNS resolving might or might not work, or you might see the address of your proxy

    Q: Why do I see an IPv6 address using the direct link ? I am on an IPv4 computer/network…
    A: Many computer systems have build-in ‘virtual adapters’ that allow browsers to navigate to IPv6 address using some remote IPv6 proxy. However, DNS resolving and using Domainnames might not work.

  • IP Tables

    Using IP Tables to prevent multiple random login attempts from hackers

    Here is a short descrition of my iptable-rules for preventing random, multiple login attemps, usually a sign of hacking-attempts. (At some point I installed a new server and had hundreds of login attempts from only two or three ip-addresses in the first 48 hours.)
    You can check this in your auth.log file using:

    cat /var/log/auth.log |grep "Failed password for"

    A more extensive how-to on IPtables can be found at the Debian-wiki. The procedure descibed there can be used to create/implement the rules described here on a permanent basis.

    First some iptables-basics (summary):
    iptables -L shows the current rules.
    iptables -F clears the current rules.

    In order to filter out random ssh login attempts the following file (/etc/iptables.test.rules) can be used:

    *filter
    -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent --set
    -N drop-and-log-it
    -A drop-and-log-it -j LOG --log-level info --log-prefix "**TOO MANY ATTEMPTS**"
    -A drop-and-log-it -j DROP
    -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent --update --seconds 120 --hitcount 2 -j drop-and-log-it
    COMMIT

    This will detect if there are two (hitcount) failed loginattempts in a time period of 120 seconds (–seconds) from the same originating IP-address. If this is the case, the procdure ‘drop-and-log’ will be executed. It will add a logentry to the messages-logfile in /var/log/messages . After this, it drops the incoming connection.

    The rules in your /etc/iptables.test.rules file are implemented using the following command:

    iptables-restore < /etc/iptables.test.rules

    Check using iptables -L if the rules are indeed there.

    The rules should work now, and if too many log in attempts are made that IP-address is blocked for 2 minutes and an entry is added to your logfile.

    [ad#Google Adsense-2]

  • Using Bind as a local DNS-proxy

    This page contains the basic instructions and links to more elaborate sites, explaining the basics of installing and starting a BIND9 DNS-caching service on your own Debian server.

    Why?
    Using your own DNS can be faster. It is also a good way of understanding how DNS works. For me, one other reason was that at some point my provider’s DNS was mailfunctioning for some hours. The network was clearly working, since i could just use IP-numbers to go to various site on the internet. I then decided to set up a DNS-cache to avoid such dependency.

    How?
    On Debian, installing Bind9 via Apt is done using:

    apt-get install bind9 dnsutils bind9-doc resolvconf ufw

    This installs BIND and some other utilities that can come in handy.
    To start bind:

    /etc/init.d/bind9 start

    After the default installation BIND will be added to the default start-up sequence

    After this the basic functionality should be installed.
    To use the local version of BIND you should editing /etc/resolv.conf , and point it to the localhost 127.0.0.1.
    BIND will automattically listen to port 53, the deafult DNS-port.
    Test this using:

    dig debian.org

    This should give domain-information about debian.org (in this example), and it should list the local server (your server) in one of the last lines.

    Example output:

    /home# dig debian.org

    ; <<>> DiG 9.5.1-P2 <<>> debian.org
    ;; global options: printcmd
    ;; Got answer:
    ;; ->>HEADER<<-- opcode: QUERY, status: NOERROR, id: 53825
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 3, ADDITIONAL: 1

    ;; QUESTION SECTION:
    ;debian.org. IN A

    ;; ANSWER SECTION:
    debian.org. 544 IN A 194.109.137.218
    debian.org. 544 IN A 128.31.0.51

    ;; AUTHORITY SECTION:

    debian.org. 1744 IN NS raff.debian.org.
    debian.org. 1744 IN NS rietz.debian.org.
    debian.org. 1744 IN NS klecker.debian.org.

    ;; ADDITIONAL SECTION:
    klecker.debian.org. 1744 IN AAAA 2001:888:2000:12::2

    ;; Query time: 1 msec
    ;; SERVER: 127.0.0.1#53(127.0.0.1)
    ;; WHEN: Wed Aug 19 16:26:40 2009
    ;; MSG SIZE rcvd: 149

    /home#

    The query time ( in one of the last lines) is probably in the order of hunderds of milliseconds for the first query.
    Repeating the same query will give a low query-time (0 or 1 ms), since the entry has now been cached locally; This is the easiest way to see that the BIND-cache works.

    For more on the configuration you can look at this more extensive 'how-to' .

    Switch Logging on:
    To switch BIND9-log on in order to see all queries in your logfiles use:

    rndc querylog

    You can then view the system log( including the DNS queries) using:

    cat /var/log/syslog

    Allow other computers to use your dns server
    To allow other computers to use your dns server, you have to add the following 'allow-query'-option in your /etc/bind/named.conf.options configuration-file ( replace by an IP address, or an IP range ):

    allow-query { SomeIP ; 127.0.0.1; };

    127.0.0.1 allows queries from your localhost. Adding other IP-addresses or ranges allows other computers to use your server. An obvious use is by adding 192.168.0.0/24, which allows other machines on your local network to use your local DNS-server.

    Block others from using your dns server
    In the same way, if the configuration is set to 'allow-query { any } ' , other can be blocked from using youe dns-server by removing 'any', and replacing it by individual IP-numbers or by for example the local IP-range.

  • Ssh-tunnel internet proxy

    Using an SSH-connection to tunnel your internet traffic through a remote server
    This is a ‘how to’ with information to use any ssh-enabled server as a gateway to the open internet from a closed network, such as a network behind an unwanted firewall.
    By using an ssh-tunnel, any portnumbers that cannot pass due to restriction of the firewall, are now ‘encapsulated’ in a tunnel through a port that is allowed to pass the firewall. By using encryption, any outsider cannot see what kind of traffic you are sending, making it hard to use ‘packet-inspection’ to filter specific applications or content.

    Why?
    If you are in a closed network (that filters or blocks some sites or protocols), or you are in a network-environment you don’t trust (for example at an internet cafe with your own laptop) you might want to route all your internettraffic through an encrypted ‘tunnelconnection’ to a trusted server outside your current domain. In this way your internettraffic can pass any firewall that is inbetween. Because the tunnel is encrypted, the content of the tunnel cannot be read, providing an anonymous way to do so.

    How?
    Setting up such an (SSL-)encrypted connection via an ssh-server consists of 2 parts:

    Part 1: Create the SSH-tunnel
    Part 2 Tell your application to use the tunnel instead of the ‘standard’ internet gateway

    What do you need ?
    - You need an ssh account on some server outside of the restricted domain.

    Part 1: Creating an SSH-tunnel from a linux-machine to a remote server

    You can use the standard ssh-command to set up a secure connection AND set-up your tunnel using the following command:
    (replace USER with your username and IPADDRESS with the IP-address of the SSH-server).

    ssh -D 4040 IPADDRESS -l USER

    You will now be prompted for your password.
    This command creates a (local) port 4040 that will act as a (local) proxy. All traffic to this port will be forwarded to IP-address.
    As long as this connection is active, one can point any application to 127.0.0.1:4040 , and the application will use this tunnel to send its IP-traffic to.

    Note that:
    - You have to remain logged-in to keep the tunnel active.
    - 4040 is just a ‘random’ port; You can use any other (unused) port, as long as you also use it in the proxy settings in the next step.
    - Make sure the port you choose is not in use by any other application on your computer.

    Part 2: Make your application use the tunnel via the local proxy at 127.0.0.1:4040

    You now need to set-up the application to use the tunnel. This can be achieved by pointing the application to the local proxy as configured in the ssh-connection using the following proxy-settings:

    Proxy-settings:

    – Choose SOCKSv5 in your proxy-settings
    – IP address : 127.0.0.1
    – Port 4040

    Note that 127.0.0.1 is the ip-address of ‘local-host’, or your own computer. Port 4040 is the port we use in this example. Any other port can be used, as long as it is free on your local machine.

    Example 1 : Mozilla Firefox / Iceweasel webbrowser

    In firefox / Iceweasel (setup is similar in many other browsers):
    – Open: ‘Edit’, ‘Preferences’, ‘Advanced’, ‘Network’
    – Choose ‘Settings’ to open the settings menu
    – Click on ‘Manual Proxy Configuration’
    – In ‘Manual Proxy’, enter : SOCKSv4, Socks-host 127.0.0.1, port 4040

    – Make sure the other entries are empty, because they override the SOCKS-settings.
    The proxy is now set and should work. Try to open any site in the browser. Check for example myipaddress.com to see from which IP you are entering the internet. It should now show the IP of the proxyserver.

    Example 2: Using the Gnome Proxy settings for all Gnome Applications (including the webbrowser)

    Gnome has the possibility to set up the proxy for all trafix via the Gnome Network Manager.
    Go to:
    – ‘System’, ‘Preferences’ on the Gnome-taskbar ( per default on the top of your Desktop).
    – Select ‘Network Proxy’.
    – Enter the SOCKS5 proxy values 127.0.0.1 , port 4040
    – Make sure the other entries are empty, because they override the SOCKS-settings.

General Linux Stuff:

  • Using OSM data for routing

    Openstreetmaps contains a lot of detailed infvormation of many parts of the world; This information is contained in an OSM-file, and can be freely used for any application.

    My goal is to perform route-calculations using Open Street Map data (i.e. determine possible routes from A to B and find the shortest path).
    There is quite some information available on the web, however some of it seems outdated a bit. ( outdated to the extend that some of the steps to take or commands to use are no longer exactly valid, although the general idea is still the same).

    These are my notes on how to get a working system that allows me to use OSM-data for determining a route, using Spatialite, an SQLITE-version adapter for use with geoinformation and spatial queries..

    Preparation steps/ installing packages
    Prerequisits, or what to do first:
    we need to install spatialite and spatialite tools.
    On Gentoo:
    emerge -av spatialite spatialite-tools
    Spatialite is now available on your system, and running “spatialite” give a similar database-terminal as when running ‘sqlite’

    Overview: Start routing.
    Since the actual routing-algorithm is performed by operations in the database some preparation is needed for the data to be ready for use in routing:
    - Get an OSM-file for an area of interest ( i.e. your hometown :-) )
    - Use ‘spatilite-tools to import the data in an sqlite/spatialite database. ( spatialite is sqlite with some additional functionalite specific for routing purposes).
    - Create the necessary routing info (again using spatialite-tools).
    - Start querying the spatialite-database, either directly ( command prompt), or using pyspatialite , a modified version of pysqlite that takes into account the extra functionality of a spatialite-database (compared to an sqlite database).

    Get an OSM-file
    - Go to Openstreetmap,
    - Select the “export”-tab ,
    - Select (either manually or by using lon/lat coordinates) an area
    - Export.

    You are only allowed to export a small area due to server limitations, but larger files can be downloaded completely at various locatios, for an overview see the download-section.
    However, when starting to use Spatialite, better start with a small area for testing purposes ( since processing whole continents takes some more processor-time).

    Import in Spatialite / sqlite database
    This step assumes you have an OSM-file called name.osm (replace name with the actual name).

    Create an sqlite-database from the osm-file:
    spatialite_osm_net -o name.osm -d .sqlite -T roads -m

    Next, create a new table containing network information in the database (note: next command is one command, and should be placed on one line in bash):
    spatialite_network -d name.sqlite -T roads -g geometry -c length -t node_to -f node_from -n name -o roads_net_data

    Last, create a table called “roads_net_tabel” with “graph”-information about the roads connection various nodes in your selected area:
    spatialite name.sqlite 'CREATE VIRTUAL TABLE "roads_net" USING VirtualNetwork("roads_net_data")'
    (again, this is one line, not two as it might appear in your browser)

    See this link about routing using spatialite, specifically the paragraph “Creating routing data”.

    Start Routing
    Start spatialite using
    spatialite name.sqlite
    Now you entered a regular sqlite-shell.

    Start routing using:
    select * from roads_net where nodefrom = 1840 AND nodeto = 2638 ;

    This simple command now queries the database for a route from node 1840 to node 2638. ( Note that the nodes I query from and to are chosen randomly by me in this example).
    Result should look something like the following:
    Algorithm|ArcRowid|NodeFrom|NodeTo|Cost|Geometry|Name
    Dijkstra||1840|2638|160.67405722996||
    Dijkstra|2377|1840|1874|13.1715757332811||
    Dijkstra|2393|1874|1865|2.71515965136783||
    Dijkstra|2400|1865|1883|2.24290239479153||
    Dijkstra|1577|1883|1916|3.68138351828062||
    Dijkstra|4646|1916|2194|45.9721562356986||
    Dijkstra|1981|2194|2537|55.9335573133834||
    Dijkstra|1977|2537|2637|35.9918700870997||
    Dijkstra|1975|2637|2638|0.96545229605724||

    If you are unsure if the nodes 1840 or 2680 exist ( fe because you selected a small OSM-area with only a few tens of nodes), test the routing using:
    select * from roads_net where nodefrom = 1 AND nodeto = 3 ;

    if somehow this doesnt return any results, ther emight have been a problem when creating the routing-info.
    Try ( in the spatialite-shell):
    spatialite> select * from roads_nodes limit 10;

    This should give something like the following, where the first column is the node-ID in this database, and the second column is the OSM-node-id:
    1|43791847|1|
    2|43800255|1|
    3|43735986|1|
    4|1493825604|2|
    5|1419273601|1|
    6|43792337|3|
    7|43768546|3|
    8|1419273597|3|
    9|43791469|3|
    10|43803689|1|

  • MCE Remote and XBMC

    Getting the Hauppage MCE USB Remote KIT to to work seems to be kind of a hassle every time a new system is installed or major changes take place.
    (date: March 2013)

    Initial notes:
    - There is no need to use LIRC. This remote has kernelmodules available and can work

      without LIRC

    - Some keys on the remote work immediately, mainy navigation ( left/right) and play and pause. However, what does not work are “OK” and “Back” ( whichis annoying since without them you cant navigate :-) )
    -So, the default keymapping does not match so changes are needed.

    Assumption and setup:
    - XBMC is installed
    - The kernelmodules are installed (mine is called “ZZ REF”).
    - Use a modern kernel ( mine is 3.8.4, but the kernelmodules for this MCE remote have been there since some time already so a somewhat older kernel should work as well.
    - UDEV 197 (so the location of devices in /dev might be different compared with older versions of udev)
    - In my case : The hardware is an early model ASROCK ION 330 (so somewhat outdated).

    Install ir-keytable
    We need to use ir-keymap to map the new keybindings:
    emerge ir-keytable

    Check if the remote is detected and works using ir-keytables:
    ir-keytable -s rc1 -t
    Note that rc1 might be another node in your case, for example rc0. Now press some puttons on the remote. This should give something similar to:
    (none) # ir-keytable -s rc1 -t
    Testing events. Please, press CTRL-C to abort.
    1364276112.820767: event MSC: scancode = 800f0410
    1364276112.820767: event key down: KEY_VOLUMEUP (0x0073)
    1364276112.820767: event sync
    1364276112.955744: event MSC: scancode = 800f0410
    1364276112.955744: event sync
    1364276113.202434: event key up: KEY_VOLUMEUP (0x0073)
    1364276113.202434: event sync

    Also, press “OK” and “BACK” and see that the codes are recognised:
    (none)# ir-keytable -s rc1 -t
    Testing events. Please, press CTRL-C to abort.
    1364276386.608950: event MSC: scancode = 800f0422
    1364276386.608950: event key down: KEY_OK (0x0160)
    1364276386.608950: event sync
    1364276386.742948: event MSC: scancode = 800f0422
    1364276386.742948: event sync
    1364276386.992447: event key up: KEY_OK (0x0160)
    1364276386.992447: event sync
    1364276388.535968: event MSC: scancode = 1f3f
    1364276388.535968: event sync
    1364276388.611966: event MSC: scancode = 800f0423
    1364276388.611966: event key down: KEY_EXIT (0x00ae)
    1364276388.611966: event sync
    1364276388.744979: event MSC: scancode = 800f0423
    1364276388.744979: event sync
    1364276388.992479: event key up: KEY_EXIT (0x00ae)
    1364276388.992479: event sync

    Now copy the “regular” keymap to a new file (so we dont have to edit the original) :
    cp /etc/rc_keymaps/rc6_mce /etc/rc_keymaps/rc6_mce_mod

    Editing the rc6_mce_mod keymapfile to allow OK and BACK
    Some of the buttons “just work”, but OK and BACK do not.
    This has to do with the way the key-presses are translated: KEY_C means its translated to the same as pressing a C on a keyboard.
    KEY_OK however has no keyboard-equivalent, so we will change the key that that specific scancode relates to to “ENTER”

    - Find the scancode to “KEY_OK”:
    0x800f0422 KEY_OK
    Change this to :
    0x800f0422 KEY_ENTER

    Quit and save the modified file and load the new keyfile using ir-keytable :
    ir-keytable -s rc1 -c -p RC-5,RC-6 -w /etc/rc_keymaps/rc6_mce_mod

    Thats it, now the OK button sends “enters” to XBMC, making it possible to select items in the menu :-)
    Check again using ir-keytable -s rc1 -t as above , pressing OK should now give “KEY_ENTER” instead of KEY_OK.

    Wake On Remote Power Button
    You can use the powerbutton to start the hardware from suspend (Note that it is nog possible to do a “cold” boot; suspend is needed first).

    echo enabled > /sys/bus/usb/devices/2-3/power/wakeup
    echo enabled > /sys/bus/usb/devices/usb2/power/wakeup

    FAQ/troubleshooting:

    Q: – IRW doesn’t show any characters at all when I press buttons ?
    A: – Well, this bothered me too… spend to many time trying to fix this, but in the end just left it (not working) since apperently my remote wirks with XBMC. No LIRC or IRW needed here….

    Q: – Using ir-keytable says “Not found device rc0″
    A: – Using the “-s” option lets you define where to look for the device. try both “-s rc0″ and -s rc1″ .
    For example, first try ir-keytable -t

    If this results in an error try some other node:

    ir-keytable -s rc1 -t
    or
    ir-keytable -s rc0 -t

  • Use PowerTop to optimize powersettings

    A handy tool for looking at your powercomumpotion is PowerTop. It gives detailed information and gives some advice about how to reduce power consumption.
    After installign powertop, start it up as root and it will give you information about all kinds of parameters, like:
    - how often is your CPU waken-up ? And by what processes?

    In order for PowerTop to use you need to have “debug Filesystem” enabled in your kernel; If not activate it under the ‘kernel hacking’-section in kernel-config, and recompile your kernel.

    Making PowerTop changes permanent

    PowerTop gives advice about some powersaving featurs which you can toggle in PowerTop under ‘tunables’. However, changes made there are not permanent, so you need to now the commands that PowerTop uses to switch-on all those features.

    Luckily, you can extract them from PowerTop: Create a powertop-advice HTML-file using:
    powertop –html

    This generates a file called ‘PowerTOP.html’, which contains the info in HTML-format so it is readable using a webbrowser.
    Basically, PowerTop uses “echo” to send a “1″ of a “y” to various devicenodes in order to activate powersaving features). So, in order to extract the commands, we grep for ‘echo’ :

    grep ‘echo’ PowerTOP.html | sed ‘s/.*\(echo.*\);.*/\1/g’

    which gives in my case:

    Mediaserver# grep ‘echo’ PowerTOP.html | sed ‘s/.*\(echo.*\);.*/\1/g’
    echo ’1′ > ‘/sys/module/snd_hda_intel/parameters/power_save’
    echo ‘min_power’ > ‘/sys/class/scsi_host/host0/link_power_management_policy’
    echo ‘auto’ > ‘/sys/bus/usb/devices/1-1/power/control’
    echo ‘auto’ > ‘/sys/bus/usb/devices/3-2/power/control’
    echo ‘auto’ > ‘/sys/bus/pci/devices/0000:00:13.0/power/control’
    echo ‘auto’ > ‘/sys/bus/pci/devices/0000:00:01.1/power/control’

    In order to make them permanent, I added them to my startup sequence by creating a file “/etc/local.d/PowerSafeSettings.start” containing the above commands.

    PS: To get them in a file:
    ” grep ‘echo’ PowerTOP.html | sed ‘s/.*\(echo.*\);.*/\1/g’ > ps.sh “

  • Home Server Power Management

    For my always-on homeserver my whish was to have low energy consumption, so next to using low-power hardware (mini-itx with an efficcient pico-PSU) I tried to find optimal powersettings.

    Some useful guides:
    Less Watts
    PowerTop
    Gentoo Power Management Guide

    Frequency control
    I use the “ondemand-governor” which reduces the cpu-frequency to the lowest values (800MHz) when not much computing-power is needed.
    There are many options, like installing a “user-space” governor”, but I simply use the on-demand governor which you can do by making it the default one in the kernel-config.

    Harddisk spin-down
    HDparm can be used to set the spin-down time of the disks; The number after the S is the number of intervals of 5 seconds; 36 gives 180 seconds or 3 minutes.
    hdparm -S36 /dev/sdb

    To check if it is really down you can either listen or use:
    /usr/sbin/smartctl -i -n standby /dev/sdb |/bin/grep mode
    which gives seomthing like this, confirming it is in stanby:
    Homeserver # /usr/sbin/smartctl -i -n standby /dev/sdb |/bin/grep mode
    Device is in STANDBY mode, exit(2)
    Homeserver #

    Ofcoarse, of you set the spindown to 3 minutes, but there is some process that needs the disk every 2 minutes, it will never sleep. Also, making the disk spin-down only makes sense if it stays idle for at least 10 minutes or more at the time, since waking it up also costs energy.
    This means the time-out time you want to use might vary from disk to disk: You might want to keep the systemdisk awake (an SSD in my case so it can’t spin down), but put the disk containing your mediafiles to sleep.

    You might want to play around with parameters and check if it actually sleeps; and if it wakes up, you want to find out what is waking it up.

    Using PowerTop
    A handy tool for looking at your powercomumpotion is PowerTop. It gives detailed information and gives some advice about how to reduce power consumption. After installin powertop, start it up as root and it will give you information about all kinds of parameters.
    See this post on how to make changes permanent.

  • TV Headend post processing of recordings

    Post processing
    A post-processing command can be configured to be run by TVHeadend after a recording has been finished.
    Recordings in TVHeadend produce an mkv file (there are some more options in the transcoding branch).
    Post-processing can be handy for example :
    -to transcode the recording to a different media-format,
    - to move a file depending on certain rules (move series one way, movies another), or
    -to create a lower-bitrate versions of a recording for watching on a mobile device like andorid or iphone, or just to save disk-space (for regular TV signals the entire stream is recorded, resulting in a large file (8 Mb/s, giving a file of 3 or 4 GBytes for an hour of recording).
    - or just to create a notification that a recording has finished, using for example a script that sends an email or a chat-message using xmpp.

    Execute a bash command or script from Tvheadend
    Scripts are called from the tab “configuration”, “digital video recorder” , and then the field “Post-processor command”

    Use a bash script for processing
    One can either execute a single line bash-command, or use that single line command to execute an external bash-script.

    Most flexible way is this last option: to point the post-processing command to a bashscript, so you can add more commands and create a more complex post-processing script. Also, you are more flexible in testing the script.
    So, first write a seperate script that does the transcoding, and than to call that script from TVHeadend.

    Example Transcoding script using FFMPEG:
    Create a file called “convert.sh” using your text editor
    nano convert.sh
    with the code below; note that the script creates a logfile in “/opt/tvheadend/home” ; This is the home-location of my thheadend-user, but this might be different for your server. Any loication can be used, as long as the tvheadend user has permissions to write there:

    #!/bin/bash
    echo "start script"
    /bin/date >>/opt/tvheadend/home/convert.log
    /usr/bin/ffmpeg -y -i "$1" -b 1000000 -ac 2 "$1.mpeg" >/opt/tvheadend/home/convert.log 2>&1

    Make the script executable by the tvheadend user ( or by all users):
    chmod +x convert.sh

    The script takes the inputfilename as option:
    ./convert.sh filename.mpeg

    What does it do?
    This simple script uses the filename specified as input for ffmpeg ($1 in the code).

    The transcoding takes place in the line “/usr/bin/ffmpeg”, before the dev/null part. You need to specifiy the complete path to ffmpeg ( or any other command you use).


    /usr/bin/ffmpeg -y -i "$1" -b 1000000 -ac 2 "$1.mpeg"

    FFmpeg takes the file $1, transcodes it to an mpeg-file with a maximum bitrate of 10000000 (-b 1000000), and saves it to the same filename with the addition of .mpeg ( “$1.mpeg”). /dev/null is needed for ffmpeg to put its output to, otherwise the script will work from the commandline (bash shell), but not when it is called from Tvheadend.

    The “>>/opt/tvheadend/home/convert.log 2>&1″ part writes the output to a logfile for debugging purposes. The logfile is located in /opt/tvheadend/home, since that is where the home directory of my Tvheadend user is located. (This might be different for your system if you used a different install method or parameters when you installed tvheadend; just make sure Tvheadend is allowed to write to that file).

    Transcoding files
    As mentioned, an obvious option for using post-processing is transcoding of the recorded file.

    Transcoding files from one format into another or from one resolution to another ( with all the options there are) is a subject in itself. There are many ways to encode sound or video (like flv, mpeg, mpeg4, flac), containerfiles (like mkv, avi or mp4) and various programs to display or create various formats. For example, an mkv-file (for example as created by a tvheadend recording) is a container, containing a video and audio streams (for example, streams for various languages), and maybe also other information and subtitles. To read about the difference between containers, mediaformats and codecs take a look at lifehacker.com/5893250/whats-the-difference-between-all-these-video-formats-and-which-one-should-i-use.
    One way to transcode a mediafile is to use ffmpeg, but others are available as well, like VLC ( the well-know media player has a powerfull command line interface for all kind of transcoding and streaking options), “handbrake” or mencoder.

    To analyse a stream one can use mediainfo. This is a commandline tool to analyse the content of a containerfile or mediafile. There is also a gui-version.
    Install MediaInfo using using:
    On gentoo:
    emerge --ask mediainfo
    Or on Debian:
    apt-get install mediainfo

  • libtiff.so.3: cannot open shared object file: No such file or directory

    A common error, especially when mixing stable (amd64) with some unstable (~amd64) packages, is that versions needed and versions present of some exotic application do not align, and sometime cannot allign since portage can only install one.
    In my case, I ran into this error using PlayOnLinux:

    lex ~ $ playonlinux
    Traceback (most recent call last):
    File "mainwindow.py", line 30, in
    import wx
    File "/usr/lib64/python2.7/site-packages/wx-2.8-gtk2-unicode/wx/__init__.py", line 45, in
    from wx._core import *
    File "/usr/lib64/python2.7/site-packages/wx-2.8-gtk2-unicode/wx/_core.py", line 4, in
    import _core_
    ImportError: libtiff.so.3: cannot open shared object file: No such file or directory
    ./playonlinux: line 96: python2.6: command not found
    lex ~ $

    So, the problem:
    libtiff.so.3 is missing !

    First, check what version is there, and what version(s) is/are not:

    cd /usr/lib
    ls -l |grep libtiff

    This give the available version of libtiff, in my case:

    lex /usr/lib $ ls -l |grep libtiff
    lrwxrwxrwx 1 root root 16 Jan 28 19:32 libtiff.so -> libtiff.so.5.0.5
    lrwxrwxrwx 1 root root 16 Jan 28 19:32 libtiff.so.5 -> libtiff.so.5.0.5
    -rwxr-xr-x 1 root root 472112 Jan 28 19:32 libtiff.so.5.0.5
    lrwxrwxrwx 1 root root 18 Jan 28 19:32 libtiffxx.so -> libtiffxx.so.5.0.5
    lrwxrwxrwx 1 root root 18 Jan 28 19:32 libtiffxx.so.5 -> libtiffxx.so.5.0.5
    -rwxr-xr-x 1 root root 10136 Jan 28 19:32 libtiffxx.so.5.0.5

    Ok, become root and create a link called ” libtiff.so.3″ to “libtiff.so.5.0.5″.
    lex lib # ln -s libtiff.so.5.0.5 libtiff.so.3

    In this whay, the program can find the right libtiff, but it actually is another one.
    Note that this *might* or *mightnot* work, depending on the exact changes inversions between the one needed and the one present. Sometimes, it just works, problems solved.

  • WSGI

    WSGI (Web Server Gateway Interface) is designed to allow communication between Python and web-applications. You can build webapplications in Python that you can call from a website, or WSGI can be used to create an entire web-framework.

    WSGI is replacing Mod_Python as a way to use Python in interactive web programming. THis page contains some simple example-scripts to start using WSGI.

    Prerequisite
    You first need to install WSGI on your server; On Debian this can be installed using apt-get install libapache2-mod-wsgi

    Getting started: Small Python/WSGI programs

    ‘Hello world’ using WSGI

    filename: helloworld.wsgi
    def application(environ, start_response):
    status = '200 OK'
    output = ' Hello World!
    '

    response_headers = [('Content-type', 'text/html'),
    ('Content-Length',str(len(output)))]
    start_response(status, response_headers)
    return [output]

    So, this looks easy: Create a file called helloworld.wsgi and place it in the wsgi-app directory (per default on Debian this is /usr/local/www/wsgi-scripts ).
    When pointing your browser to the right location + filename, the function ‘application’ is executed.
    Some strings are defined (status, output), the start_response command outputs both content-type and content-length, and finally the output is returned as if it where a ‘real’ html file.

    ‘Hello world Extended’ using WSGI
    Now, to get some information from the webserver to the application we extend the program using wsgiref.util.

    filename: helloworld2.wsgi
    import wsgiref.util
    def application(environ, start_response):
    status = '200 OK'
    output1 = ' Hello World!
    '
    output2 = 'requested URL = '+ wsgiref.util.request_uri(environ, include_query=0)
    output = output1 + output2
    response_headers = [('Content-type', 'text/html'),
    ('Content-Length',str(len(output)))]

    start_response(status, response_headers)
    return [output]

    Show IP of webpage visitor using WSGI

    import wsgiref.util
    #from paste.request import parse_formvars

    def application(environ, start_response):
    status = '200 OK'
    headers = [('Content-type', 'text/plain')]

    start_response(status, headers)

    IP_adres =environ.get('REMOTE_ADDR')

    start_response('200 OK', [('content-type', 'text/html')])

    Header = ''' Your IP is:

    '''

    End_tekst '''

    To see your IP v6 address, connect to this page using itsIPv6 address

    '''

    #return [environ.get('REMOTE_ADDR')]
    return [Header + IP_adres+End_tekst]

  • Network Tools

    Show network Trafic / bandwidth usage

    There are various tools to get an idea of the current bandwidth usage oon your system for the various network adpaters you might have.
    A simple tool is Iftop; You can also use IPTRAF.
    A short introduction to both is given down here:

    IFTOP
    ‘iftop’ is a fairly simple tool that analyses network traffic on your network interface and gives bandwidths for up and downlink traffic.
    Install by using (gentoo) :
    # emerge --ask -v iftop
    This should give something like:
    MediaTestServer ~ # emerge --ask -v iftop
    These are the packages that would be merged, in order:
    Calculating dependencies... done!
    [ebuild N ] net-libs/libpcap-1.1.1-r1 USE="-bluetooth -ipv6 -libnl -static-libs" 569 kB
    [ebuild N ] net-analyzer/iftop-0.17 157 kB
    Total: 2 packages (2 new), Size of downloads: 725 kB
    Would you like to merge these packages? [Yes/No] y

    Yes, of course :-)

    Now, to get an idea of the trafic on your system, run iftop:
    # iftop
    interface: eth0
    IP address is: 192.168.0.30
    MAC address is: 00:1a:92:03:e0:a1

    IPTRAF
    IPTraf (describing itself as a “Interactive Colorful IP LAN Monitor” indeed has some more colors ( allthough not that much more) and is a commandline interactive frontend for <> that gives various forms of network information including bandwidths and TCP and UDP info.
    Install (emerge) using:
    # emerge --ask -v iptraf-ng

    Giving:

    MediaTestServer ~ # emerge --ask -v iptraf-ng
    These are the packages that would be merged, in order:
    Calculating dependencies... done!
    [ebuild N ] net-analyzer/iptraf-ng-1.0.2 USE="-static-libs" 857 kB
    Total: 1 package (1 new), Size of downloads: 857 kB
    Would you like to merge these packages? [Yes/No] y
    >>> Verifying ebuild manifests
    >>> Emerging (1 of 1) net-analyzer/iptraf-ng-1.0.2
    >>> Downloading 'http://distfiles.gentoo.org/distfiles/iptraf-ng-1.0.2.tar.gz'
    --2012-01-22 17:25:26-- http://distfiles.gentoo.org/distf

    And just start using the following command give a lot of real-time infomration about network usage, allowing debugging or determining the load caused by specific system processes:

    MediaTestServer ~ # iptraf

  • Python Image manipulation

    Python can be used to manipulate or create pictures. One way is to use PIL , the Python Image Library. This page contains some simple examples to get started.

    For the examples below this image will be used for testing the apps. Download the image and save it as “Linux_Logo.png” in the same directory as the python scripts ( or change the filename in the script).

    Example 1: Read values .png file per pixel
    This program determines the size of the picture, and then reads and prints the values of each pixel:


    from PIL import Image
    png_file_name='Linux_Logo.png'
    # change to the filename. use full path if the png-file is not in the same dir.

    def MAIN():
    im = Image.open(png_file_name)
    pix = im.load()
    image_size= im.size
    print 'image size is ' , image_size
    xrange = image_size[0] # set the x-range of the image from the image size.
    yrange = image_size[1]

    for j in range (0,xrange):
    for i in range(0,yrange):
    value = pix[j,i]
    print value

    MAIN()

    Example 2: Read values and change first component (red in RGB) to 255
    This program results in the logo to be transformed to a more ‘red-coloured’ picture.
    It reads all pixels, changes the value of the first byte in RGB ( RED) to 255, and keeps the other colours.
    The result should be something like this:


    from PIL import Image
    png_file_name='Linux_Logo.png'
    # change the filename to the picture you want to use.

    newData = list()

    def MAIN():
    img = Image.open(png_file_name)
    pix = img.load()
    image_size= img.size
    print 'image size is ' , image_size
    yrange = image_size[0]
    xrange = image_size[1]

    for j in range (0,xrange):
    for i in range(0,yrange):
    value = pix[i,j]
    new_value= (255,value[1] , value[2],value[3])
    #Example:
    #Make first (Red in RGB) byte 255 ( maxvalue)
    #keep other components what they were.
    newData.append(new_value)

    img.putdata(newData)
    img.save( "result.png")

    MAIN()

  • System specs

    Hdparm: Show Disk read write performance

    One can use hdparm to get an idea for harddisk or ssd reads using:

    # hdparm -t /dev/sda
    Typical output can be:
    /dev/sda:
    Timing buffered disk reads: 328 MB in 3.02 seconds = 108.74 MB/sec

    This gives a starting point when benchmarking your system, and determining bottlenecks in performance. Actual read and write times might be different, since hdparm reads a continuous part of the harddisk / solid state disk , and ‘real’ usage usually consists of many random reads of different parts of the disk

    Dmicode: Show memory type and settingsd
    To show RAM-type (DDR,DDR2, DDR3 etc) and actual bus speed use:
    #dmidecode --type 17
    Output:
    # dmidecode 2.9
    SMBIOS 2.4 present.

    Handle 0x002C, DMI type 17, 27 bytes
    Memory Device
    Array Handle: 0x002A
    Error Information Handle: Not Provided
    Total Width: 64 bits
    Data Width: 72 bits
    Size: 1024 MB
    Form Factor: DIMM
    Set: None
    Locator: DIMM0
    Bank Locator: BANK0
    Type: DDR2
    Type Detail: Synchronous
    Speed: 667 MHz (1.5 ns)
    Manufacturer: Manufacturer0
    Serial Number: SerNum0
    Asset Tag: AssetTagNum0
    Part Number: PartNum0

    Handle 0x002E, DMI type 17, 27 bytes
    Memory Device
    Array Handle: 0x002A
    Error Information Handle: Not Provided
    Total Width: 64 bits
    Data Width: 72 bits
    Size: 1024 MB
    Form Factor: DIMM
    Set: None
    Locator: DIMM1
    Bank Locator: BANK1
    Type: DDR2
    Type Detail: Synchronous
    Speed: 667 MHz (1.5 ns)
    Manufacturer: Manufacturer1
    Serial Number: SerNum1
    Asset Tag: AssetTagNum1
    Part Number: PartNum1

    Show actual Ethernet Linkmode
    As root run
    ethtool eth0

    To get detailed information about the supported link modes, and the actual used speedmode of your ethernet connection.
    Typical output:

    root@mediasystem:/home/lex# ethtool eth0
    Settings for eth0:
    Supported ports: [ MII ]
    Supported link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Full
    Supports auto-negotiation: Yes
    Advertised link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Full
    Advertised pause frame use: No
    Advertised auto-negotiation: Yes
    Speed: 1000Mb/s
    Duplex: Full
    Port: MII
    PHYAD: 3
    Transceiver: external
    Auto-negotiation: on
    Supports Wake-on: g
    Wake-on: d
    Link detected: yes

  • Use HDHomerun with TVHeadend

    Use HDHomerun with TVHeadend via a virtual device /dev/dvb/adapter1

    The HDHomerun from SiliconDust is a network-attached dual DVB-C/T tuner. It allows you to watch tv from various PC’s or devices in your home. Standard usage is to use VLC or some device to connect to the “networktuner”, and no device /dev/dvb/ is needed. However, it is needed to allow usage with TVHeadend or SASC-NG. (MythTV has standard support for the HDHomerun, but in my country some ISPs ( mine) encypt all digital tv cable signals; Since there is no ‘standard’ CAM-option in Mythtv, I need a virtual device to decrypt).

    Luckely, there is an opensource Linux-driver dvbhdhomerun created by Villyft on Sourceforge on to connect the HDHomerun and create a virtual DVB-device on your local system.

    Prerequisits
    - A working HDHomerun device; The assumption is that the device works with the standard Silicondust Software.
    - TVHeadend installed
    - Optional: OSCam or similar if you need decryption.

    Note: Linux Kernel >3.0 doesnt work with older versions of dvbhdhomerun
    (12 october 2012)
    There have been significant changes in the DVB-part of the linux kernel lately.
    Therefor, for newer Linux-kernels, ( 3.0 and higher) you need the latest version of dvbhdhomerun, 0.11.

    Get the drivers

    First create a directory to download the files and to install the software in /opt/dvbhdhomerun :
    mkdir /opt/dvbhdhomerun
    cd /opt/dvbhdhomerun

    The driver consists of a kernel part and the userspace part.
    Next, get the Drivers (both kernel and userspace part in one package) from sourcefourge and unpack it:

    wget http://downloads.sourceforge.net/project/dvbhdhomerun/dvbhdhomerun_0.0.9.tar.gz
    tar -xvf dvbhdhomerun_0.0.9.tar.gz
    cd dvbhdhomerun-0.0.9/

    Building and installing the kernel part
    cd kernel/
    make
    make-install

    Now, if everything worked you should be able to load the kernelmodule that is created:
    modprobe dvb_hdhomerun

    Building and installing the Userspace part
    First enter the directory containing the userpart:
    cd /opt/dvbhdhomerun-0.0.9/userhdhomerun

    Here, open CMAKELists.txt and change the line with “LIBHDHOMERUN_PATH” to point to ‘libhdhomerun’, the official driver as provided by SiliconDust, so that the userspace program knows where to find it.

    MediaServer userhdhomerun # nano CMakeLists.txt
    Then, ‘make’ the program:

    MediaServer userhdhomerun # make

    Get the drivers
    Now run the program to test it using:

    make run
    This instance should keep running, so open another terminal and see if the newly created virtual DVB-adapters show up! ( they should ;-) )


    MediaServer # cd /dev/
    MediaServer dev # ls |grep hdhome
    hdhomerun_control
    hdhomerun_data0
    hdhomerun_data1
    MediaServer dev #

    and :
    MediaServer dev # cd dvb/
    MediaServer dvb # ls
    adapter0 adapter1

    FAQ/Troubleshooting:
    Villyft added a page with tests to perform, like using the regular w_scan tools as described here

    PROBLEM: HDHomeRun uses ATSC
    What: w_scan shows “ATSC “HDHomeRun ATSC”: Good” , but i need DVB-Cor DVB-T. (or: My system think I have ATSC, but I live in Europe and actually have DVB-C or DVB-T).

    Output includes:
    843000: QAM256(time: 16:18)
    849000: QAM256(time: 16:22)

    ERROR: Sorry - i couldn't get any working frequency/transponder
    Nothing to scan!!

    Solution: You need to tell the dvbhdhomrun-driver what type of network you have. If not, it assumes the default, which is ATSC. More in this is here.
    Change the congigurationfile in /etc/dvbhdhomerun

    MediaTestServer dev # cat /etc/dvbhdhomerun
    # Types can any of:
    # DVB-C
    # DVB-T
    # ATSC
    # You can change the tuner type for each tuner on your HDHomeRun by
    # exchanging the 1210E3DC-X with the serial number of your tuner:

    #[1210E3DC-0]
    [12104502-0]
    tuner_type=DVB-T
    #[1210E3DC-1]
    [1210456602-1]
    #tuner_type=DVB-C
    tuner_type=DVB-T

    warnings during compile
    WARNING: "dvb_frontend_detach" [/opt/dvbhdhomerun/dvbhdhomerun/kernel/dvb_hdhomerun.ko] undefined!
    WARNING: "dvb_dmxdev_release" [/opt/dvbhdhomerun/dvbhdhomerun/kernel/dvb_hdhomerun.ko] undefined!
    WARNING: "dvb_dmx_swfilter" [/opt/dvbhdhomerun/dvbhdhomerun/kernel/dvb_hdhomerun.ko] undefined!
    WARNING: "dvb_dmx_release" [/opt/dvbhdhomerun/dvbhdhomerun/kernel/dvb_hdhomerun.ko] undefined!
    WARNING: "dvb_register_adapter" [/opt/dvbhdhomerun/dvbhdhomerun/kernel/dvb_hdhomerun.ko] undefined!
    WARNING: "dvb_dmxdev_init" [/opt/dvbhdhomerun/dvbhdhomerun/kernel/dvb_hdhomerun.ko] undefined!

    This probably means the DVB-stuff is not build into the kernel as it should be.
    Chachk the configuration of your kernelcompile, and make sure config_dvb_core is set:

    CONFIG_DVB_CORE=m

    (thanks to Villyft for pointing this out to me after I made this mistake….)

    PROBLEM: There is no file /etc/dvbhdhomerun
    The file is not created by default. The is one in /opt/hdhomerun/etc/dvbhdhomerun.
    Copy that one and edit it to meet your situation:

    cp /opt/hdhomerun/etc/dvbhdhomerun /etc/dvbhdhomerun
    nano /opt/hdhomerun/etc/dvbhdhomerun

    PROBLEM: W-scan: nothing to scan!!
    What: W_Scan gives an error, nothing to scan:
    W_SCAN ERROR: Sorry - i couldn't get any working frequency/transponder / Nothing to scan!!

    Using w_scan, for exmple like this for scanning ATSC device, gives “nothing to scan”:

    # w_scan -f a -A 3 -c US -o 7 -x


    w_scan version 20110702 (compiled for DVB API 5.2)
    using settings for UNITED STATES
    ATSC
    VSB US/CA, DVB-T TW
    QAM US/CA
    frontend_type ATSC, channellist 2
    output format initial tuning data
    WARNING: could not guess your codepage. Falling back to 'UTF-8'
    output charset 'UTF-8', use -C to override
    Info: using DVB adapter auto detection.
    /dev/dvb/adapter0/frontend0 -> DVB-T "DiBcom 7000PC": specified was ATSC -> SEARCH NEXT ONE.
    /dev/dvb/adapter1/frontend0 -> ATSC "HDHomeRun ATSC": good :-)
    /dev/dvb/adapter2/frontend0 -> ATSC "HDHomeRun ATSC": good :-)
    Using ATSC frontend (adapter /dev/dvb/adapter1/frontend0)
    -_-_-_-_ Getting frontend capabilities-_-_-_-_
    Using DVB API 5.4
    frontend 'HDHomeRun ATSC' supports
    INVERSION_AUTO
    8VSB
    16VSB
    QAM_64
    QAM_256
    FREQ (54.00MHz ... 858.00MHz)
    -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_
    57000: 8VSB(time: 00:00)
    63000: 8VSB(time: 00:04)
    69000: 8VSB(time: 00:07) ^C
    843000: QAM256(time: 16:18)
    849000: QAM256(time: 16:22)

    ERROR: Sorry - i couldn't get any working frequency/transponder
    Nothing to scan!!

    Answer:

    Are you scanning for the right service-type?
    Look at the info directly after “Info: using DVB adapter auto detection.”.
    - Are all your devices there?
    - And is the correct servicetype detected? (possibilities are: DVB-C, DVB-T and ATSC).

    If your device is detected, but show up as the wrong type (f.e. ATSC when you actually have DVB-C) you might have a faulty (or none) configuration file, see Q1 above to specify service type in /etc/dvbhdhomerun

    Note that there are three in this example, since the first one is a pinnacle USB 72e dvb-t device that allready was configured and attached to the system!

  • Kernel module for Phoenix Smargo Smartreader+

    For my HTPC system I use the Phoenix Smargo Smartreader+ in combination with OScam to be able to watch also the encrypted DVB-T channels. For the USB smartreader to work the USB FTDI Serial Driver must be compiled in the kernel ( AFAIK as ‘build-in”, not as a module).

    Selecting the right Driver in the kernel

    The Driver “USB FTDI Single Port Serial Driver” can be selected in the kernel configuration ( recently tried at least the following kernels: Linux kernel 3.0.5, 3.1.5, and 3.2.0 and similar) at:

    <*> --> USB Support
    <*> --> USB Serial Converter Support


    And select “USB FTDI Single Port Serial Driver”

    Troubleshooting:
    ÖSCam give an error “Error: cannot open ttyusb0
    This means either the device is not there at all, or OSCam is looking at the wrong place.

    1) First, always check if the reader is present using ls /dev abd see of there is any ttyusbx device.
    2) If it is not there the driver is probably not compiled for the current kernel.

  • Installing OSCam

    Installing OSCam with the Phoenix Smargo Smartreader.

    OScam is an open source CAM Server-program that can access your subsription card (via a cardreader) and ‘serves’ the Keys to your TV-decryption program, for example TVHeadend.
    In my case I use OSCAM with the Smargo Smartreader to watch Digitenne DVB-T signals.

    The system I use:
    - AMD Athlon 4800+
    - Gentoo Linux kernel 3.2.0
    - TVHeadend

    Prerequisits
    * Working DVB-tuner ( in my case I use both the Pinnacle 72e single tuner, and the HDHomerun dual networktuner with Digitenne)
    * You need “USB FTDI Single Port Serial Driver” selected in your kernel The cardreader should be visible in /dev, for example at /dev/ttyUSB0.

    Get and install the code
    You can get the code for OSCam using SVN from http://streamboard.gmc.to/svn/oscam:
    svn co http://streamboard.gmc.to/svn/oscam /opt/oscam/src/

    There is an excelent ‘how-to’ on installing the code on lonelycoder.com/ .
    Following those steps install OSCAM in /opt/OSCAM.
    The instruction are general since there is no actual integration with a specific distro.

    Configuring OSCAM for use with Digitenne/DVB-T
    For using Oscam with another operator (in this example: Digitenne, CAID 0B00) we need to change the relevant parameters in the configuration files.
    The Digitenne CAID is 0B00 ( but will be different for your operator)

    The configuration files are located in in /opt/oscam/etc/
    In my case there are three configurationfiles:

    oscam.conf
    oscam.server
    oscam.user

    To start it is adviced to use ONLY those three;
    Adding more config files adds extra complexity (but gives more configuration options).
    This makes troubleshooting harder since the configuration files can contradict each other, and it will not immideately be clear which setting is used. So, best way is to first get it all running with only those three files.

    Here are my configurationfiles for Digitenne ( Dutch DVB-T provider, CAID 0B00)
    oscam.conf:

    [global]
    usrfile = /opt/oscam/oscamuser.log
    logfile = /opt/oscam/oscam.log
    cwlogdir = /opt/oscam/etc/cw
    disablelog = 0
    disableuserfile = 0
    usrfileflag = 0
    clienttimeout = 2000
    fallbacktimeout = 1500
    clientmaxidle = 120
    failbantime = 0
    bindwait = 120
    netprio = 0
    clientdyndns = 0
    resolvedelay = 2
    unlockparental = 0
    nice = -1
    serialreadertimeout = 1000
    maxlogsize = 10
    waitforcards = 1
    preferlocalcards = 1
    saveinithistory = 0
    readerrestartseconds = 5
    lb_mode = 0
    lb_save = 0
    lb_nbest_readers = 1
    lb_nfb_readers = 1
    lb_min_ecmcount = 5
    lb_max_ecmcount = 500
    lb_reopen_seconds = 900
    resolvegethostbyname = 0

    [newcamd]
    #port = 15050@0604:000000
    port = 15050@0B00:000000

    key = 0102030405060708091011121314
    allowed =
    keepalive = 1
    mgclient = 0

    [webif]
    httpport = 15080
    httpuser = username
    httppwd = password
    httprefresh = 0
    httpallowed = 127.0.0.1,192.168.0.107
    httphideidleclients = 0
    httpreadonly = 0

    oscam.server

    # reader configuration

    [reader]
    label =r eader1
    protocol = mouse
    detect = CD
    device = /dev/ttyUSB0
    group =
    emmcache = 1,3,2
    services = services1
    caid = 0B00
    mhz = 500
    cardmhz = 500

    oscam.user:

    [account]
    user = tvheadend
    pwd = tvheadend
    uniq = 0
    group = 1
    ident = 0B00:0
    caid = 0B00
    au = Digitenne

    Troubleshooting

    Q: The cardreader changes node from /dev/USB0 to /dev/USB1
    A: Unplugging and replugging the USB-cardreader might place the reader at another port, for example ttyUSB1, which will make OScam unable to find it.
    However, after reboot the cardreader should be back at /dev/ttyUSB0, so for a ‘working’ system where the device is not unplugged there should be no such problems.

    Q: The cardreader is not visible in /dev !
    A: You have to select support for this device in the kernel. In my case for the Smargo Smart Reader the driver is “USB FTDI Single Port Serial Driver”

  • Hello world!

    Welcome to WordPress. This is your first post. Edit or delete it, then start blogging!

  • WordPress after Debian Upgrade from Lenny to Squeeze

    WordPress and phpmyadmin fail on upgrade Lenny to Squeeze
    When I upgraded Debian Lenny to Squeeze, WordPress failed, or at least it stopped serving webpages. Some of the php-features did not work out of the box after upgrading (which might be not so strange given the huge changes and upgrades available in Squeeze).
    In my case, this effected both WordPress and phpmyadmin: Both wouldn’t show any pages after upgrading, but apache and MySQL where running. Since you probably want to get WordPress back up running quickly after upgrading, down here a solution to this problem.

    After upgrading Lenny to Squeeze
    The behaviour has to do (as far as I can see) with the php-auth-http module needed for WordPress and phpmyadmin.
    This can be solved by installing the following packages, afgter upgrading to Squeeze:

    apt-get install php-auth-http php5-cgi libapache2-mod-php5

    Now, restart Apache ( /etc/init.d/apache2 restart), and that was it.
    Wordpress, phpmyadmin ( and probably other php-stuff) work again (at least in my case)

    Prerequisite:
    Here the assumption is the upgrade of the rest of the system is complete (including upgrading Apache2 and upgrading to the new Mysql version 5.1) using the following:
    (more on this in the upgrade guide at the Debian website, see the release note, chapter 4 )

    apt-get update
    apt-get upgrade
    apt-get dist-upgrade

    [ad#Google Adsense-2]

  • Password protecting a website using .htaccess

    Protecting a webpage on your server using .htaccess

    There are two steps to this:
    1. Create a passwords-file containing the passwords of the people you would like to be able to login.
    2. Tell apache , using .htaccess. to only give access to people using a password.

    1. Creating password file

    First define a directory where you wnat to put your password-file. This file has to be OUTSIDE your www-domain, since otherwise people might get hold of it.
    I use a bewly made directory
    /var/wwwpasswords/

    Go there ( cd /var/wwwpasswords/ ) and use the following command andchange ‘user’ to the username yo uwant to add to the passowordfile:

    htpasswd -c .webpagepasswords user

    If you want to add a user to an existing passwordfile use:

    htpasswd .webpagepasswords anotheruser

    This create a file called ‘webpagepasswords’ containing the password of the user ‘user’.

    2. Add a .htaccess file in the web-dir you want to protect

    Go to the dir you want to protect, for example ‘/var/www/intranet/’ , and use a text-editor ( like nano ) to create/edit .htaccess:

    nano .htaccess

    now add the following :

    AuthUserFile /var/wwwpasswords/.webpagepasswords
    AuthType Basic
    AuthName "You need to loging to view this page"
    Require valid-user

    This does it; from now on the webpage , www.-yourdomain-.-yourcountry-/intranet is protected and you need a password to login.

  • MyIP

    What is my IP address?
    To see you IP address ( v4 or v6) click down here:

    Show IP

    What is my IPv6 address?
    The above link *should* work using either IPv6 or IPv4 addresses. However, if you see an IPv4 address, and where expecting an IPv6 address, you can try activating the script using the IPv6 address of this page.

    Show IP Using IPv6 address
    [2001:4b98:dc0:41:216:3eff:fea8:e73d]

    Q: Why don’t I see an IPv6 address ?
    A: Depending on how you enter the IPv6 web, you might see either an IPv6 or an IP v4 address. If you are part of the IPv6 internet you should see your IPv6 address. However, if you use IPv6 via an IPv4-2-IPv6 adapter, or by ‘tunneling’ IPv6 traffic over an IPv4 network to the IPv6 internet via some IPv6 ready proxy, DNS resolving might or might not work, or you might see the address of your proxy

    Q: Why do I see an IPv6 address using the direct link ? I am on an IPv4 computer/network…
    A: Many computer systems have build-in ‘virtual adapters’ that allow browsers to navigate to IPv6 address using some remote IPv6 proxy. However, DNS resolving and using Domainnames might not work.

  • IP Tables

    Using IP Tables to prevent multiple random login attempts from hackers

    Here is a short descrition of my iptable-rules for preventing random, multiple login attemps, usually a sign of hacking-attempts. (At some point I installed a new server and had hundreds of login attempts from only two or three ip-addresses in the first 48 hours.)
    You can check this in your auth.log file using:

    cat /var/log/auth.log |grep "Failed password for"

    A more extensive how-to on IPtables can be found at the Debian-wiki. The procedure descibed there can be used to create/implement the rules described here on a permanent basis.

    First some iptables-basics (summary):
    iptables -L shows the current rules.
    iptables -F clears the current rules.

    In order to filter out random ssh login attempts the following file (/etc/iptables.test.rules) can be used:

    *filter
    -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent --set
    -N drop-and-log-it
    -A drop-and-log-it -j LOG --log-level info --log-prefix "**TOO MANY ATTEMPTS**"
    -A drop-and-log-it -j DROP
    -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent --update --seconds 120 --hitcount 2 -j drop-and-log-it
    COMMIT

    This will detect if there are two (hitcount) failed loginattempts in a time period of 120 seconds (–seconds) from the same originating IP-address. If this is the case, the procdure ‘drop-and-log’ will be executed. It will add a logentry to the messages-logfile in /var/log/messages . After this, it drops the incoming connection.

    The rules in your /etc/iptables.test.rules file are implemented using the following command:

    iptables-restore < /etc/iptables.test.rules

    Check using iptables -L if the rules are indeed there.

    The rules should work now, and if too many log in attempts are made that IP-address is blocked for 2 minutes and an entry is added to your logfile.

    [ad#Google Adsense-2]

  • Using Bind as a local DNS-proxy

    This page contains the basic instructions and links to more elaborate sites, explaining the basics of installing and starting a BIND9 DNS-caching service on your own Debian server.

    Why?
    Using your own DNS can be faster. It is also a good way of understanding how DNS works. For me, one other reason was that at some point my provider’s DNS was mailfunctioning for some hours. The network was clearly working, since i could just use IP-numbers to go to various site on the internet. I then decided to set up a DNS-cache to avoid such dependency.

    How?
    On Debian, installing Bind9 via Apt is done using:

    apt-get install bind9 dnsutils bind9-doc resolvconf ufw

    This installs BIND and some other utilities that can come in handy.
    To start bind:

    /etc/init.d/bind9 start

    After the default installation BIND will be added to the default start-up sequence

    After this the basic functionality should be installed.
    To use the local version of BIND you should editing /etc/resolv.conf , and point it to the localhost 127.0.0.1.
    BIND will automattically listen to port 53, the deafult DNS-port.
    Test this using:

    dig debian.org

    This should give domain-information about debian.org (in this example), and it should list the local server (your server) in one of the last lines.

    Example output:

    /home# dig debian.org

    ; <<>> DiG 9.5.1-P2 <<>> debian.org
    ;; global options: printcmd
    ;; Got answer:
    ;; ->>HEADER<<-- opcode: QUERY, status: NOERROR, id: 53825
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 3, ADDITIONAL: 1

    ;; QUESTION SECTION:
    ;debian.org. IN A

    ;; ANSWER SECTION:
    debian.org. 544 IN A 194.109.137.218
    debian.org. 544 IN A 128.31.0.51

    ;; AUTHORITY SECTION:

    debian.org. 1744 IN NS raff.debian.org.
    debian.org. 1744 IN NS rietz.debian.org.
    debian.org. 1744 IN NS klecker.debian.org.

    ;; ADDITIONAL SECTION:
    klecker.debian.org. 1744 IN AAAA 2001:888:2000:12::2

    ;; Query time: 1 msec
    ;; SERVER: 127.0.0.1#53(127.0.0.1)
    ;; WHEN: Wed Aug 19 16:26:40 2009
    ;; MSG SIZE rcvd: 149

    /home#

    The query time ( in one of the last lines) is probably in the order of hunderds of milliseconds for the first query.
    Repeating the same query will give a low query-time (0 or 1 ms), since the entry has now been cached locally; This is the easiest way to see that the BIND-cache works.

    For more on the configuration you can look at this more extensive 'how-to' .

    Switch Logging on:
    To switch BIND9-log on in order to see all queries in your logfiles use:

    rndc querylog

    You can then view the system log( including the DNS queries) using:

    cat /var/log/syslog

    Allow other computers to use your dns server
    To allow other computers to use your dns server, you have to add the following 'allow-query'-option in your /etc/bind/named.conf.options configuration-file ( replace by an IP address, or an IP range ):

    allow-query { SomeIP ; 127.0.0.1; };

    127.0.0.1 allows queries from your localhost. Adding other IP-addresses or ranges allows other computers to use your server. An obvious use is by adding 192.168.0.0/24, which allows other machines on your local network to use your local DNS-server.

    Block others from using your dns server
    In the same way, if the configuration is set to 'allow-query { any } ' , other can be blocked from using youe dns-server by removing 'any', and replacing it by individual IP-numbers or by for example the local IP-range.

  • Shell commands

    Using TAR to PACK and COMPRESS files or directory

    This is a simple ‘how to’/ reminder of packing and unpackling using tar…

    Using TAR to extract or create tar.gz files:

    To create a .tgz file:

    tar -czvf example.tgz dir

    To unpack and uncompress a tar.gz file, use (with the ‘z’-option):

    tar -xzvf example.tgz

    To unpack a tar.bz2 file, use: (with the ‘j’-option):
    tar -jxvf filename.tar.bz2

    Using MOUNT to mount an ISO image of a cd or DVD

    When you have an ISO for a CD or DVD, you can either burn the image to a physical CD or DVD, or mount the image directly .
    This is done using :

    mount -o loop isofile.iso /mnt/cd

    This mounts the image to /mnt/cd
    /mnt/cd must exist first. Mounting must be done as root, however depending on permissions other users can use the image after mounting.

  • MythTV: Firefox as default browser

    Using Firefox (Iceweasel) as the default browser for MythTV

    This page describes the configuration to set up Firefox as the default browser of MythTV, replacing the ‘ internal’ browser.
    Just changing the browser was the easy part, however I had a bit more trouble setting up the remote control for Firefox. In the end, (assuming Lirc works for Mythtv) setting up the remote to work with Firefox in Mythtv is not so hard, but I had to use some small tricks to get it working properly, and there was no step-by-step guide to get it working, so I decided to write down this small ‘howto’.

    There are two parts of this setup:
    Preparation: Starting Firefox from a shellscript.
    Create simple startup-scripts that will be used later on.

    A. Seting up Remote Control for Firefox (Iceweasel) using LIRC
    This allows you to start and control Firefox using a remote control.

    B. Using Firefox with remote control as the browser for MythTV
    This will make firefox the default browser for Mythtv.

    Prerequisite:
    The assumption is that you have lircd and mythtv installed and running.

    Preparation: Starting Firefox from a shellscript.
    We will use a shell-script to start Firefox since we want to execute some specific commands when Firefox is started from within Mythtv. Use nano/vi/anyeditor to create a file called firefox-start.sh, and put the following into the file:

    #!/bin/sh
    /usr/bin/firefox $1 --fullscreen &
    exit 0

    Then create a file firefox-stop.sh. killall is used since Iceweasel keeps running after returning to mythtv:
    sudo killall firefox-bin &

    Use:
    ~$ chmod 777 firefox-start.sh
    ~$ chmod 777 firefox-stop.sh

    to make the file executable.
    Test the script by issuing the following command from the prompt in the directory where the scipt is placed:
    ~$ ./firefox-start.sh

    This should start Firefox. If this works we are no ready to configure .lirc to allow execution ofthis script using keypresses on the remote control.

    A. Seting up Remote Control for Firefox (Iceweasel) using LIRC

    You can use the scripts to start firefox we have to define Remote Control and LIRC for firefox.

    Remote Control and LIRC for Firefox
    To start programs using the RC you can use ‘irexec’, and to send commands to a program use ‘irxevent’.
    We need to configure both, using IREXEC to start/stop Firefox, and using IRXEVENT to send commands to the browser once it is running.
    Both irexec and irxevent look at your .lirc file for key-bindings/definitions, so we first have to tell LIRC what to do when a button is pressed by adding the following to the .lirc file in your home directory ( in this example : /home/mediauser/.lirc ).

    begin
    prog = irexec
    button = Blue
    config = /home/mediauser/firefox-start.sh
    end

    begin
    prog = irexec
    button = Back
    repeat = 3
    config = /home/mediauser/firefox-stop.sh
    end

    begin
    prog = irxevent
    button = ChanUp
    repeat = 3
    config = Key ctrl-plus CurrentWindow
    end

    begin
    prog = irxevent
    button = ChanDown
    repeat = 3
    config = Key ctrl-minus CurrentWindow
    end

    This programs four buttons on your remote, more buttons can be added later:
    - the blue button starts ‘Firefox-start.sh’
    - the ‘back’ button starts ‘Firefox-stop.sh’
    - Channel up is equivalent to ctrl-plus, zooming in
    - Channel down is equivalent to ctrl-minus, zooming out

    Note that the first two are buttons for irexec, since they are used to start and stop the application, and the last two are buttons for irxevent since they represent buttons to be send to the running program in focus.

    Execute IREXEC and IRXEVENT in daemon-mode, replacing /home/mediauser with the path to your .lircrc-file:
    ~$ irexec -d /home/mediauser/.lircrc
    ~$ irxevent -d /home/mediauser/.lircrc

    This should do the trick; if no errors occurred both irexec and irxevent are listening.
    To test: Press the blue button on the remote, and Firefox should start. Press ‘back’ and Firefox should stop.

    Troubleshooting: What if this doesn’t work:
    - First make sure lircd is already running
    - Sometimes old instances of irexec or irxevents are still running, and the remote is still configured using the old mappings: Use
    killall irexec
    killall irxevent

    to stop them, and verify using ps -A to make sure they are all killed.Then try again to execute irexec and irxevent.

    B. Using Firefox with remote control as the browser for MythTV

    Using Firefox as your browser in Mythtv
    To change the browser, open Mythtv and go to ‘ setup’ , ‘ web’. There you find the command that starts the browser. The default is ‘internal’. You can change this to ‘/usr/bin/firefox %URL%’ to start firefox. However, we will use the shellscript we created earlier on since we want to add some specific command when firefox is started.

    /home/mediauser/firefox-start.sh %URL%

    The %URL%-part is needed to add the page mythtv wants to open to the command being executed.

    Switching on and off irexec when using Firefox

    Irxevent sends commands (like ‘up’ and ‘ down’) to any program in the active window. Somehow, when activated, it adds up to the commands already received by Mythtv from the .lirc-part of the file configered for mythtv. (that is, the part with prog=mythtv sends the command once, and the part with prog=irxevent sends the command another time). Consequence is that pressing ‘up’ once becomes equivalent to two times ‘arrow up’, which is unwanted (you would skip every other menu option :-) ). This can be solved by starting irexec and irxevents just before calling firefox, and by stopping irexec and irxevents just after stopping firefox.
    This can be achieved by adding the corresponding commands to the shell scripts for stopping and starting firefox.
    firefox-start.sh becomes:

    irexec -d /home/mediauser/.lircrc
    irxevent -d /home/mediauser/.lircrc

    #!/bin/sh
    /usr/bin/firefox $1 --fullscreen &
    exit 0

    firefox-stop.sh becomes:

    killall firefox-bin &
    killall irexec
    killall irxevent

    Contact
    If you have any remarks, questions or suggestion please send me an e-mail at: info[at]lex-web.net

  • Ssh-tunnel internet proxy

    Using an SSH-connection to tunnel your internet traffic through a remote server
    This is a ‘how to’ with information to use any ssh-enabled server as a gateway to the open internet from a closed network, such as a network behind an unwanted firewall.
    By using an ssh-tunnel, any portnumbers that cannot pass due to restriction of the firewall, are now ‘encapsulated’ in a tunnel through a port that is allowed to pass the firewall. By using encryption, any outsider cannot see what kind of traffic you are sending, making it hard to use ‘packet-inspection’ to filter specific applications or content.

    Why?
    If you are in a closed network (that filters or blocks some sites or protocols), or you are in a network-environment you don’t trust (for example at an internet cafe with your own laptop) you might want to route all your internettraffic through an encrypted ‘tunnelconnection’ to a trusted server outside your current domain. In this way your internettraffic can pass any firewall that is inbetween. Because the tunnel is encrypted, the content of the tunnel cannot be read, providing an anonymous way to do so.

    How?
    Setting up such an (SSL-)encrypted connection via an ssh-server consists of 2 parts:

    Part 1: Create the SSH-tunnel
    Part 2 Tell your application to use the tunnel instead of the ‘standard’ internet gateway

    What do you need ?
    - You need an ssh account on some server outside of the restricted domain.

    Part 1: Creating an SSH-tunnel from a linux-machine to a remote server

    You can use the standard ssh-command to set up a secure connection AND set-up your tunnel using the following command:
    (replace USER with your username and IPADDRESS with the IP-address of the SSH-server).

    ssh -D 4040 IPADDRESS -l USER

    You will now be prompted for your password.
    This command creates a (local) port 4040 that will act as a (local) proxy. All traffic to this port will be forwarded to IP-address.
    As long as this connection is active, one can point any application to 127.0.0.1:4040 , and the application will use this tunnel to send its IP-traffic to.

    Note that:
    - You have to remain logged-in to keep the tunnel active.
    - 4040 is just a ‘random’ port; You can use any other (unused) port, as long as you also use it in the proxy settings in the next step.
    - Make sure the port you choose is not in use by any other application on your computer.

    Part 2: Make your application use the tunnel via the local proxy at 127.0.0.1:4040

    You now need to set-up the application to use the tunnel. This can be achieved by pointing the application to the local proxy as configured in the ssh-connection using the following proxy-settings:

    Proxy-settings:

    – Choose SOCKSv5 in your proxy-settings
    – IP address : 127.0.0.1
    – Port 4040

    Note that 127.0.0.1 is the ip-address of ‘local-host’, or your own computer. Port 4040 is the port we use in this example. Any other port can be used, as long as it is free on your local machine.

    Example 1 : Mozilla Firefox / Iceweasel webbrowser

    In firefox / Iceweasel (setup is similar in many other browsers):
    – Open: ‘Edit’, ‘Preferences’, ‘Advanced’, ‘Network’
    – Choose ‘Settings’ to open the settings menu
    – Click on ‘Manual Proxy Configuration’
    – In ‘Manual Proxy’, enter : SOCKSv4, Socks-host 127.0.0.1, port 4040

    – Make sure the other entries are empty, because they override the SOCKS-settings.
    The proxy is now set and should work. Try to open any site in the browser. Check for example myipaddress.com to see from which IP you are entering the internet. It should now show the IP of the proxyserver.

    Example 2: Using the Gnome Proxy settings for all Gnome Applications (including the webbrowser)

    Gnome has the possibility to set up the proxy for all trafix via the Gnome Network Manager.
    Go to:
    – ‘System’, ‘Preferences’ on the Gnome-taskbar ( per default on the top of your Desktop).
    – Select ‘Network Proxy’.
    – Enter the SOCKS5 proxy values 127.0.0.1 , port 4040
    – Make sure the other entries are empty, because they override the SOCKS-settings.

  • Mod Python

    Note: Although mod_python is still available, there are now (2010 and later) other ways of executing python from an Apache webserver.
    These days the preferred method of doing so is using WSGI.


    Python is a practical scripting-language that is easy to learn and easy to use.
    Together with Mod_python as an Apache-module it can be used to execute server-side python-scripts from webpage, allowing for interactive websites.
    This page contains a number of small mod_python programs I used as an ‘exercise’ to find out how to use python combined with apache to create webpages with (server-side) applications.

    The first two programs are basic examples and simple checks to see if the mod_python module works. They use pyhton to create an HTML-page as an output, and form simple ‘Hello World’ programs for Mod_python.

    The other applications involve user-input via HTML-post, and using the input in the actual application/calculation performed in the python-program.


    1. Simple ‘Hello World’ program.

    Code:

    def index():
    s = ''' Hello World, written in Pyhton'''
    return s

    2. Another ‘Hello’ program, now including three choices.
    Depending on the part in the URL behind hello.py/, anouther function in the program is executed.
    ‘hello.py/’ executes the ‘def index()’-function, ‘hello.py/earth’ executes the ‘def earth()’ function, and hello.py/world execures the ‘def world()’-function in the code.

    Code:


    # hello world. Depending on the url-part behind 'hello.py' one gets either the default or World or Earth
    s = "

    Hello %s!

    "
    def index():
    return s % '!'

    def world():
    return s % 'World'

    def earth():
    return s % 'Earth'

    3. A simple ‘calculator’ for multiplying integer numbers.

    Code:

    def fill():
    s = """\

    This is a simple, online, server-side multiplication-application using mod_python.

    First number:
    Second number:



    """
    return s

    def show(req):
    cijfera = int(req.form.getfirst('cijfer1','0'))
    cijferb = int(req.form.getfirst('cijfer2','0'))
    Answer = cijfera * cijferb
    outputstring = """\

    The product of the two numbers is %s

    Let's perform another calculation...


    """
    return outputstring % Answer

    4. Another one, now whit a choice of operations: Mulitply, add, or devide.

    Code:

    def index():
    outputstring = """\

    Yet another server-side online 'calculator' in Python,
    ` with a large choise of operations to be performed ;-)

    Integer 1:
    Integer 2:

    Multiply

    Add

    Divide


    """
    return outputstring

    def show(req):
    Oper = req.form.getfirst('operatie','0')
    cijfera = int(req.form.getfirst('cijfer1','1'))
    cijferb = int(req.form.getfirst('cijfer2','1'))
    getal = 8
    if Oper == 'Mult':
    Answer = cijfera * cijferb
    if Oper == 'Add':
    Answer = cijfera + cijferb
    if Oper == 'Div':
    Answer = cijfera / cijferb

    outputstring = """\

    The anser is.... %s

    Again

    """
    return outputstring % Answer

    5. Determine IP Address of viewer: This program returns the IP address from where the webpage was requested using mod_python.

    Code:

    from mod_python import apache
    def index(req):
    req.add_common_vars()
    IPAddress = req.subprocess_env['REMOTE_ADDR']
    port = req.subprocess_env['REMOTE_PORT']
    Path = req.subprocess_env['PATH']
    htmlstring = '

    Your IP Address is ' + IPAddress + ' Port: ' + port + ''
    return htmlstring

    6. FiboOnline calculates Fibonaccie-numbers up to a certain number.

    Code:


    import cgi
    def index():
    s = """\

    This application calculates Fibonacci-sequence.

    Fibo-numbers until
    Number of Fibo-numbers in sequence

    Input:


    """
    return s

    def fib1(n): # return Fibonacci series up to n
    result = []
    a, b = 0, 1
    while b < n:
    result.append(b)
    a, b = b, a+b
    return result

    def fib2(n):
    result = []
    a, b = 0, 1
    for i in range (n):
    result.append(b)
    print b
    a, b = b, a+b
    return result

    def show(req):
    Oper = req.form.getfirst('operatie','Grens')
    cijfera = req.form.getfirst('cijfer','10')
    getal1 = int(cijfera)
    if Oper == 'Grens':
    result = fib1(getal1)
    if Oper == 'Aantal':
    result = fib2(getal1)
    print "test print "
    outputstring = """\

    The requested sequence is: %s

    back

    """
    return outputstring % result

    7. Another Fibonaccie-generator, this one plots a graph of the calculated numbers.

    Code:


    import cgi, matplotlib
    matplotlib.use('Agg')
    from matplotlib import *
    from pylab import *

    def index():
    s = """\
    This application calculates and plots the Fibonacci sequence up to the given limit.

    Fibonacci up to:
    Upper bound

    Number of elements


    """
    return s

    def fib1(n): # return Fibonacci series up to n
    result = []
    a, b = 0, 1
    while b < n:
    result.append(b)
    a, b = b, a+b
    return result

    def fib2(n):
    result = []
    a, b = 0, 1
    for i in range (n):
    result.append(b)
    a, b = b, a+b
    return result

    def PlotGrafiekje(fibreeks,xbereik):
    from matplotlib import *
    x = []
    # y = fibreeks
    for i in range(len(fibreeks)):
    x.append(i)
    scatter(x, fibreeks, s=100)
    plot (x,fibreeks)
    #save the image to hardcopy
    savefig('/var/www/www2/modpythonprogs/fibowithgraph/fiboplot')

    def show(req):
    result = []
    # getal = 0
    Oper = req.form.getfirst('operatie','Grens')
    cijfer = 3
    cijfer = req.form.getfirst('grenswaarde','10')
    getal = int(cijfer)
    if Oper == 'Grens':
    resultfromfib = fib1(getal)
    if Oper == 'Aantal':
    resultfromfib = fib2(getal)
    PlotGrafiekje(resultfromfib, getal)
    s = """\

    De uitkomst is: "%s"

    again.


    """

    return s % resultfromfib

    Another example is this on-line photo-book I use for showing my pictures online. The program scans the photo-directory, and dynamically create the html-code needed to displays the photo’s. Easy to use in your own homepage.

    [ad#Google Adsense-2]

  • Unlock Lacie Network Space 2;allow ssh

    Unlocking the Lacie Network Space 2 to allow ssh login.

    The Lacie home storage ethernet disk is an ideal device for storing and sharing information within your local network, and to make this information accessible from the outside. However, having shell access gives you the ability to execute other programs and small scripts from this ‘always on’-machine, which makes it even better! (I know you could also install a complete server, but the good thing is that power consumption is low, and you can use it to wake up other machines in your house if needed).

    System:
    These instruction worked for me on the following system:
    Lacie Network Space 2
    1 TB
    Configuration: Version 1.2.5 (note: updated, the opriginal firmware was older.)

    How?
    SSH has the possibility to allow users to login based on a shared key consisting of a public and a private part. The trick is to create a key-pair, and get half of the key-pair on the lacie, and the other part on your computer. To do this a ‘loophole’ in the lacie-security that allows a php-script to execute and safe files to the disk on the lacie is used.

    Use the script that is here ( under ‘Enabling SSH without disassembling’) :
    Enabling_ssh_scipt

    Instructions are in the header of the script as commends ; follow the steps indicated there.
    The instruction in the script are for windoz, some small comments on the instruction in case you use a linux machine:
    Basically, you do :

    Create keypair using ssh-keygen.

    lex@lexweb:~$ ssh-keygen
    Generating public/private rsa key pair.
    Enter file in which to save the key (/home/lex/.ssh/id_rsa): my_lacie_key
    Enter passphrase (empty for no passphrase):
    Enter same passphrase again:
    Your identification has been saved in my_lacie_key.
    Your public key has been saved in my_lacie_key.pub.
    The key fingerprint is:
    21:ab:13:f7:55:76:r9:77:88:e9:22:a7:75:73

    ssh-keygen will ask for a keyname, in the case of this exzample ‘my_lacie_key’.
    ssh-keygen will generate two files, my_lacie_key and my_lacie_key.pub.
    The last one contains the public key needed. Paste this key in the $ssh-key variable and place the sceipt on the openshare of your Lacie.

    - Execute the script by browsing to the file ( http:///webdav/OpenShare/ssh.php )
    The script will add the public-key to the lacie, and your computer is ‘paired’ with the lacie.

    Now, login using ssh as root and make further adjustments !

    After you follow the steps described above and in the sciptfile you should have access as root from the computer you created the ssh-code on.

    NOTE If this method works depends on what Lacie you have, and on its firmware. The newest versions allow for execution of the php script. On Older Lacie-disks it was possible to manually insert a cronjob that allowed for root access, see the following link. This option is no longer possible, at least not on the system I have (see specs): this blog

    Create keypair using ssh-keygen.
    The example is for use with putty (from windows). When doing this from a linux machine (debian) you can generate the code using ‘ssh-keygen’. It is then stored (default) in the .ssh subdirectory of your user. You have to paste the public key ( stored in .ssh/id_rsa.pub) in the php script as described in the instructions. Paste the entire content of the .ssh/id_rsa.pub-file.

    Execute the Php script to copy the public part to your Lacie Network space
    After placing the php scipt on the public share point your browser to the file.

    Note:Somehow, when using IceWeasel the file was not executed ! I actually had to use firefox on my work-laptop to execute the script.

    login using root
    After this you can log in from the machine that you created the keypair on, using:
    ssh 192.168.0.210 -l root

    (change the IP to your local address of the lacie device):

    You should now be in a shell environment on your lacie. No passwords needed since you provided the keypair manually.
    Next you can set a root password to allow login from other machines.
    Also, you might want to add an extra user for regular loging to the system (not root) by issuing adduser.

    Securing Root Login:
    Since we want to make the ‘lacie-homeserver’ available from the outside world via portporwarding on my router we need to secure the system.
    Therefore, additional security is needed since there might be many ‘random login attampts’. However, per default sudo is not available and su does not work for any loged in user.
    The sshd_config file can be used to restrict access to certain users.

    /etc/sshd_config

    By adding the following lines to the end of the file, one only allows root-login from any IP on the subdomain 192.168 (my home network). Once you add any allowuser line to this file, all users that can log in have to be specified. I added a user ‘lex’ that is allowed to login from anywhere.

    # SECURITY LINES ADDED BY LEX. REMOVE FOLLOWING LINES TO RESTORE:
    ALLOWUSERS lex@*
    ALLOWUSERS root@192.168.*

    - The AllowUser tells what user is allowed to login
    - Using 192.168.* for root makes sure root login is only permitted from your local network.

HTPC Stuff:

  • MCE Remote and XBMC

    Getting the Hauppage MCE USB Remote KIT to to work seems to be kind of a hassle every time a new system is installed or major changes take place.
    (date: March 2013)

    Initial notes:
    - There is no need to use LIRC. This remote has kernelmodules available and can work

      without LIRC

    - Some keys on the remote work immediately, mainy navigation ( left/right) and play and pause. However, what does not work are “OK” and “Back” ( whichis annoying since without them you cant navigate :-) )
    -So, the default keymapping does not match so changes are needed.

    Assumption and setup:
    - XBMC is installed
    - The kernelmodules are installed (mine is called “ZZ REF”).
    - Use a modern kernel ( mine is 3.8.4, but the kernelmodules for this MCE remote have been there since some time already so a somewhat older kernel should work as well.
    - UDEV 197 (so the location of devices in /dev might be different compared with older versions of udev)
    - In my case : The hardware is an early model ASROCK ION 330 (so somewhat outdated).

    Install ir-keytable
    We need to use ir-keymap to map the new keybindings:
    emerge ir-keytable

    Check if the remote is detected and works using ir-keytables:
    ir-keytable -s rc1 -t
    Note that rc1 might be another node in your case, for example rc0. Now press some puttons on the remote. This should give something similar to:
    (none) # ir-keytable -s rc1 -t
    Testing events. Please, press CTRL-C to abort.
    1364276112.820767: event MSC: scancode = 800f0410
    1364276112.820767: event key down: KEY_VOLUMEUP (0x0073)
    1364276112.820767: event sync
    1364276112.955744: event MSC: scancode = 800f0410
    1364276112.955744: event sync
    1364276113.202434: event key up: KEY_VOLUMEUP (0x0073)
    1364276113.202434: event sync

    Also, press “OK” and “BACK” and see that the codes are recognised:
    (none)# ir-keytable -s rc1 -t
    Testing events. Please, press CTRL-C to abort.
    1364276386.608950: event MSC: scancode = 800f0422
    1364276386.608950: event key down: KEY_OK (0x0160)
    1364276386.608950: event sync
    1364276386.742948: event MSC: scancode = 800f0422
    1364276386.742948: event sync
    1364276386.992447: event key up: KEY_OK (0x0160)
    1364276386.992447: event sync
    1364276388.535968: event MSC: scancode = 1f3f
    1364276388.535968: event sync
    1364276388.611966: event MSC: scancode = 800f0423
    1364276388.611966: event key down: KEY_EXIT (0x00ae)
    1364276388.611966: event sync
    1364276388.744979: event MSC: scancode = 800f0423
    1364276388.744979: event sync
    1364276388.992479: event key up: KEY_EXIT (0x00ae)
    1364276388.992479: event sync

    Now copy the “regular” keymap to a new file (so we dont have to edit the original) :
    cp /etc/rc_keymaps/rc6_mce /etc/rc_keymaps/rc6_mce_mod

    Editing the rc6_mce_mod keymapfile to allow OK and BACK
    Some of the buttons “just work”, but OK and BACK do not.
    This has to do with the way the key-presses are translated: KEY_C means its translated to the same as pressing a C on a keyboard.
    KEY_OK however has no keyboard-equivalent, so we will change the key that that specific scancode relates to to “ENTER”

    - Find the scancode to “KEY_OK”:
    0x800f0422 KEY_OK
    Change this to :
    0x800f0422 KEY_ENTER

    Quit and save the modified file and load the new keyfile using ir-keytable :
    ir-keytable -s rc1 -c -p RC-5,RC-6 -w /etc/rc_keymaps/rc6_mce_mod

    Thats it, now the OK button sends “enters” to XBMC, making it possible to select items in the menu :-)
    Check again using ir-keytable -s rc1 -t as above , pressing OK should now give “KEY_ENTER” instead of KEY_OK.

    Wake On Remote Power Button
    You can use the powerbutton to start the hardware from suspend (Note that it is nog possible to do a “cold” boot; suspend is needed first).

    echo enabled > /sys/bus/usb/devices/2-3/power/wakeup
    echo enabled > /sys/bus/usb/devices/usb2/power/wakeup

    FAQ/troubleshooting:

    Q: – IRW doesn’t show any characters at all when I press buttons ?
    A: – Well, this bothered me too… spend to many time trying to fix this, but in the end just left it (not working) since apperently my remote wirks with XBMC. No LIRC or IRW needed here….

    Q: – Using ir-keytable says “Not found device rc0″
    A: – Using the “-s” option lets you define where to look for the device. try both “-s rc0″ and -s rc1″ .
    For example, first try ir-keytable -t

    If this results in an error try some other node:

    ir-keytable -s rc1 -t
    or
    ir-keytable -s rc0 -t

  • Home Server Power Management

    For my always-on homeserver my whish was to have low energy consumption, so next to using low-power hardware (mini-itx with an efficcient pico-PSU) I tried to find optimal powersettings.

    Some useful guides:
    Less Watts
    PowerTop
    Gentoo Power Management Guide

    Frequency control
    I use the “ondemand-governor” which reduces the cpu-frequency to the lowest values (800MHz) when not much computing-power is needed.
    There are many options, like installing a “user-space” governor”, but I simply use the on-demand governor which you can do by making it the default one in the kernel-config.

    Harddisk spin-down
    HDparm can be used to set the spin-down time of the disks; The number after the S is the number of intervals of 5 seconds; 36 gives 180 seconds or 3 minutes.
    hdparm -S36 /dev/sdb

    To check if it is really down you can either listen or use:
    /usr/sbin/smartctl -i -n standby /dev/sdb |/bin/grep mode
    which gives seomthing like this, confirming it is in stanby:
    Homeserver # /usr/sbin/smartctl -i -n standby /dev/sdb |/bin/grep mode
    Device is in STANDBY mode, exit(2)
    Homeserver #

    Ofcoarse, of you set the spindown to 3 minutes, but there is some process that needs the disk every 2 minutes, it will never sleep. Also, making the disk spin-down only makes sense if it stays idle for at least 10 minutes or more at the time, since waking it up also costs energy.
    This means the time-out time you want to use might vary from disk to disk: You might want to keep the systemdisk awake (an SSD in my case so it can’t spin down), but put the disk containing your mediafiles to sleep.

    You might want to play around with parameters and check if it actually sleeps; and if it wakes up, you want to find out what is waking it up.

    Using PowerTop
    A handy tool for looking at your powercomumpotion is PowerTop. It gives detailed information and gives some advice about how to reduce power consumption. After installin powertop, start it up as root and it will give you information about all kinds of parameters.
    See this post on how to make changes permanent.

  • TV Headend post processing of recordings

    Post processing
    A post-processing command can be configured to be run by TVHeadend after a recording has been finished.
    Recordings in TVHeadend produce an mkv file (there are some more options in the transcoding branch).
    Post-processing can be handy for example :
    -to transcode the recording to a different media-format,
    - to move a file depending on certain rules (move series one way, movies another), or
    -to create a lower-bitrate versions of a recording for watching on a mobile device like andorid or iphone, or just to save disk-space (for regular TV signals the entire stream is recorded, resulting in a large file (8 Mb/s, giving a file of 3 or 4 GBytes for an hour of recording).
    - or just to create a notification that a recording has finished, using for example a script that sends an email or a chat-message using xmpp.

    Execute a bash command or script from Tvheadend
    Scripts are called from the tab “configuration”, “digital video recorder” , and then the field “Post-processor command”

    Use a bash script for processing
    One can either execute a single line bash-command, or use that single line command to execute an external bash-script.

    Most flexible way is this last option: to point the post-processing command to a bashscript, so you can add more commands and create a more complex post-processing script. Also, you are more flexible in testing the script.
    So, first write a seperate script that does the transcoding, and than to call that script from TVHeadend.

    Example Transcoding script using FFMPEG:
    Create a file called “convert.sh” using your text editor
    nano convert.sh
    with the code below; note that the script creates a logfile in “/opt/tvheadend/home” ; This is the home-location of my thheadend-user, but this might be different for your server. Any loication can be used, as long as the tvheadend user has permissions to write there:

    #!/bin/bash
    echo "start script"
    /bin/date >>/opt/tvheadend/home/convert.log
    /usr/bin/ffmpeg -y -i "$1" -b 1000000 -ac 2 "$1.mpeg" >/opt/tvheadend/home/convert.log 2>&1

    Make the script executable by the tvheadend user ( or by all users):
    chmod +x convert.sh

    The script takes the inputfilename as option:
    ./convert.sh filename.mpeg

    What does it do?
    This simple script uses the filename specified as input for ffmpeg ($1 in the code).

    The transcoding takes place in the line “/usr/bin/ffmpeg”, before the dev/null part. You need to specifiy the complete path to ffmpeg ( or any other command you use).


    /usr/bin/ffmpeg -y -i "$1" -b 1000000 -ac 2 "$1.mpeg"

    FFmpeg takes the file $1, transcodes it to an mpeg-file with a maximum bitrate of 10000000 (-b 1000000), and saves it to the same filename with the addition of .mpeg ( “$1.mpeg”). /dev/null is needed for ffmpeg to put its output to, otherwise the script will work from the commandline (bash shell), but not when it is called from Tvheadend.

    The “>>/opt/tvheadend/home/convert.log 2>&1″ part writes the output to a logfile for debugging purposes. The logfile is located in /opt/tvheadend/home, since that is where the home directory of my Tvheadend user is located. (This might be different for your system if you used a different install method or parameters when you installed tvheadend; just make sure Tvheadend is allowed to write to that file).

    Transcoding files
    As mentioned, an obvious option for using post-processing is transcoding of the recorded file.

    Transcoding files from one format into another or from one resolution to another ( with all the options there are) is a subject in itself. There are many ways to encode sound or video (like flv, mpeg, mpeg4, flac), containerfiles (like mkv, avi or mp4) and various programs to display or create various formats. For example, an mkv-file (for example as created by a tvheadend recording) is a container, containing a video and audio streams (for example, streams for various languages), and maybe also other information and subtitles. To read about the difference between containers, mediaformats and codecs take a look at lifehacker.com/5893250/whats-the-difference-between-all-these-video-formats-and-which-one-should-i-use.
    One way to transcode a mediafile is to use ffmpeg, but others are available as well, like VLC ( the well-know media player has a powerfull command line interface for all kind of transcoding and streaking options), “handbrake” or mencoder.

    To analyse a stream one can use mediainfo. This is a commandline tool to analyse the content of a containerfile or mediafile. There is also a gui-version.
    Install MediaInfo using using:
    On gentoo:
    emerge --ask mediainfo
    Or on Debian:
    apt-get install mediainfo

  • Use HDHomerun with TVHeadend

    Use HDHomerun with TVHeadend via a virtual device /dev/dvb/adapter1

    The HDHomerun from SiliconDust is a network-attached dual DVB-C/T tuner. It allows you to watch tv from various PC’s or devices in your home. Standard usage is to use VLC or some device to connect to the “networktuner”, and no device /dev/dvb/ is needed. However, it is needed to allow usage with TVHeadend or SASC-NG. (MythTV has standard support for the HDHomerun, but in my country some ISPs ( mine) encypt all digital tv cable signals; Since there is no ‘standard’ CAM-option in Mythtv, I need a virtual device to decrypt).

    Luckely, there is an opensource Linux-driver dvbhdhomerun created by Villyft on Sourceforge on to connect the HDHomerun and create a virtual DVB-device on your local system.

    Prerequisits
    - A working HDHomerun device; The assumption is that the device works with the standard Silicondust Software.
    - TVHeadend installed
    - Optional: OSCam or similar if you need decryption.

    Note: Linux Kernel >3.0 doesnt work with older versions of dvbhdhomerun
    (12 october 2012)
    There have been significant changes in the DVB-part of the linux kernel lately.
    Therefor, for newer Linux-kernels, ( 3.0 and higher) you need the latest version of dvbhdhomerun, 0.11.

    Get the drivers

    First create a directory to download the files and to install the software in /opt/dvbhdhomerun :
    mkdir /opt/dvbhdhomerun
    cd /opt/dvbhdhomerun

    The driver consists of a kernel part and the userspace part.
    Next, get the Drivers (both kernel and userspace part in one package) from sourcefourge and unpack it:

    wget http://downloads.sourceforge.net/project/dvbhdhomerun/dvbhdhomerun_0.0.9.tar.gz
    tar -xvf dvbhdhomerun_0.0.9.tar.gz
    cd dvbhdhomerun-0.0.9/

    Building and installing the kernel part
    cd kernel/
    make
    make-install

    Now, if everything worked you should be able to load the kernelmodule that is created:
    modprobe dvb_hdhomerun

    Building and installing the Userspace part
    First enter the directory containing the userpart:
    cd /opt/dvbhdhomerun-0.0.9/userhdhomerun

    Here, open CMAKELists.txt and change the line with “LIBHDHOMERUN_PATH” to point to ‘libhdhomerun’, the official driver as provided by SiliconDust, so that the userspace program knows where to find it.

    MediaServer userhdhomerun # nano CMakeLists.txt
    Then, ‘make’ the program:

    MediaServer userhdhomerun # make

    Get the drivers
    Now run the program to test it using:

    make run
    This instance should keep running, so open another terminal and see if the newly created virtual DVB-adapters show up! ( they should ;-) )


    MediaServer # cd /dev/
    MediaServer dev # ls |grep hdhome
    hdhomerun_control
    hdhomerun_data0
    hdhomerun_data1
    MediaServer dev #

    and :
    MediaServer dev # cd dvb/
    MediaServer dvb # ls
    adapter0 adapter1

    FAQ/Troubleshooting:
    Villyft added a page with tests to perform, like using the regular w_scan tools as described here

    PROBLEM: HDHomeRun uses ATSC
    What: w_scan shows “ATSC “HDHomeRun ATSC”: Good” , but i need DVB-Cor DVB-T. (or: My system think I have ATSC, but I live in Europe and actually have DVB-C or DVB-T).

    Output includes:
    843000: QAM256(time: 16:18)
    849000: QAM256(time: 16:22)

    ERROR: Sorry - i couldn't get any working frequency/transponder
    Nothing to scan!!

    Solution: You need to tell the dvbhdhomrun-driver what type of network you have. If not, it assumes the default, which is ATSC. More in this is here.
    Change the congigurationfile in /etc/dvbhdhomerun

    MediaTestServer dev # cat /etc/dvbhdhomerun
    # Types can any of:
    # DVB-C
    # DVB-T
    # ATSC
    # You can change the tuner type for each tuner on your HDHomeRun by
    # exchanging the 1210E3DC-X with the serial number of your tuner:

    #[1210E3DC-0]
    [12104502-0]
    tuner_type=DVB-T
    #[1210E3DC-1]
    [1210456602-1]
    #tuner_type=DVB-C
    tuner_type=DVB-T

    warnings during compile
    WARNING: "dvb_frontend_detach" [/opt/dvbhdhomerun/dvbhdhomerun/kernel/dvb_hdhomerun.ko] undefined!
    WARNING: "dvb_dmxdev_release" [/opt/dvbhdhomerun/dvbhdhomerun/kernel/dvb_hdhomerun.ko] undefined!
    WARNING: "dvb_dmx_swfilter" [/opt/dvbhdhomerun/dvbhdhomerun/kernel/dvb_hdhomerun.ko] undefined!
    WARNING: "dvb_dmx_release" [/opt/dvbhdhomerun/dvbhdhomerun/kernel/dvb_hdhomerun.ko] undefined!
    WARNING: "dvb_register_adapter" [/opt/dvbhdhomerun/dvbhdhomerun/kernel/dvb_hdhomerun.ko] undefined!
    WARNING: "dvb_dmxdev_init" [/opt/dvbhdhomerun/dvbhdhomerun/kernel/dvb_hdhomerun.ko] undefined!

    This probably means the DVB-stuff is not build into the kernel as it should be.
    Chachk the configuration of your kernelcompile, and make sure config_dvb_core is set:

    CONFIG_DVB_CORE=m

    (thanks to Villyft for pointing this out to me after I made this mistake….)

    PROBLEM: There is no file /etc/dvbhdhomerun
    The file is not created by default. The is one in /opt/hdhomerun/etc/dvbhdhomerun.
    Copy that one and edit it to meet your situation:

    cp /opt/hdhomerun/etc/dvbhdhomerun /etc/dvbhdhomerun
    nano /opt/hdhomerun/etc/dvbhdhomerun

    PROBLEM: W-scan: nothing to scan!!
    What: W_Scan gives an error, nothing to scan:
    W_SCAN ERROR: Sorry - i couldn't get any working frequency/transponder / Nothing to scan!!

    Using w_scan, for exmple like this for scanning ATSC device, gives “nothing to scan”:

    # w_scan -f a -A 3 -c US -o 7 -x


    w_scan version 20110702 (compiled for DVB API 5.2)
    using settings for UNITED STATES
    ATSC
    VSB US/CA, DVB-T TW
    QAM US/CA
    frontend_type ATSC, channellist 2
    output format initial tuning data
    WARNING: could not guess your codepage. Falling back to 'UTF-8'
    output charset 'UTF-8', use -C to override
    Info: using DVB adapter auto detection.
    /dev/dvb/adapter0/frontend0 -> DVB-T "DiBcom 7000PC": specified was ATSC -> SEARCH NEXT ONE.
    /dev/dvb/adapter1/frontend0 -> ATSC "HDHomeRun ATSC": good :-)
    /dev/dvb/adapter2/frontend0 -> ATSC "HDHomeRun ATSC": good :-)
    Using ATSC frontend (adapter /dev/dvb/adapter1/frontend0)
    -_-_-_-_ Getting frontend capabilities-_-_-_-_
    Using DVB API 5.4
    frontend 'HDHomeRun ATSC' supports
    INVERSION_AUTO
    8VSB
    16VSB
    QAM_64
    QAM_256
    FREQ (54.00MHz ... 858.00MHz)
    -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_
    57000: 8VSB(time: 00:00)
    63000: 8VSB(time: 00:04)
    69000: 8VSB(time: 00:07) ^C
    843000: QAM256(time: 16:18)
    849000: QAM256(time: 16:22)

    ERROR: Sorry - i couldn't get any working frequency/transponder
    Nothing to scan!!

    Answer:

    Are you scanning for the right service-type?
    Look at the info directly after “Info: using DVB adapter auto detection.”.
    - Are all your devices there?
    - And is the correct servicetype detected? (possibilities are: DVB-C, DVB-T and ATSC).

    If your device is detected, but show up as the wrong type (f.e. ATSC when you actually have DVB-C) you might have a faulty (or none) configuration file, see Q1 above to specify service type in /etc/dvbhdhomerun

    Note that there are three in this example, since the first one is a pinnacle USB 72e dvb-t device that allready was configured and attached to the system!

  • Kernel module for Phoenix Smargo Smartreader+

    For my HTPC system I use the Phoenix Smargo Smartreader+ in combination with OScam to be able to watch also the encrypted DVB-T channels. For the USB smartreader to work the USB FTDI Serial Driver must be compiled in the kernel ( AFAIK as ‘build-in”, not as a module).

    Selecting the right Driver in the kernel

    The Driver “USB FTDI Single Port Serial Driver” can be selected in the kernel configuration ( recently tried at least the following kernels: Linux kernel 3.0.5, 3.1.5, and 3.2.0 and similar) at:

    <*> --> USB Support
    <*> --> USB Serial Converter Support


    And select “USB FTDI Single Port Serial Driver”

    Troubleshooting:
    ÖSCam give an error “Error: cannot open ttyusb0
    This means either the device is not there at all, or OSCam is looking at the wrong place.

    1) First, always check if the reader is present using ls /dev abd see of there is any ttyusbx device.
    2) If it is not there the driver is probably not compiled for the current kernel.

  • Installing OSCam

    Installing OSCam with the Phoenix Smargo Smartreader.

    OScam is an open source CAM Server-program that can access your subsription card (via a cardreader) and ‘serves’ the Keys to your TV-decryption program, for example TVHeadend.
    In my case I use OSCAM with the Smargo Smartreader to watch Digitenne DVB-T signals.

    The system I use:
    - AMD Athlon 4800+
    - Gentoo Linux kernel 3.2.0
    - TVHeadend

    Prerequisits
    * Working DVB-tuner ( in my case I use both the Pinnacle 72e single tuner, and the HDHomerun dual networktuner with Digitenne)
    * You need “USB FTDI Single Port Serial Driver” selected in your kernel The cardreader should be visible in /dev, for example at /dev/ttyUSB0.

    Get and install the code
    You can get the code for OSCam using SVN from http://streamboard.gmc.to/svn/oscam:
    svn co http://streamboard.gmc.to/svn/oscam /opt/oscam/src/

    There is an excelent ‘how-to’ on installing the code on lonelycoder.com/ .
    Following those steps install OSCAM in /opt/OSCAM.
    The instruction are general since there is no actual integration with a specific distro.

    Configuring OSCAM for use with Digitenne/DVB-T
    For using Oscam with another operator (in this example: Digitenne, CAID 0B00) we need to change the relevant parameters in the configuration files.
    The Digitenne CAID is 0B00 ( but will be different for your operator)

    The configuration files are located in in /opt/oscam/etc/
    In my case there are three configurationfiles:

    oscam.conf
    oscam.server
    oscam.user

    To start it is adviced to use ONLY those three;
    Adding more config files adds extra complexity (but gives more configuration options).
    This makes troubleshooting harder since the configuration files can contradict each other, and it will not immideately be clear which setting is used. So, best way is to first get it all running with only those three files.

    Here are my configurationfiles for Digitenne ( Dutch DVB-T provider, CAID 0B00)
    oscam.conf:

    [global]
    usrfile = /opt/oscam/oscamuser.log
    logfile = /opt/oscam/oscam.log
    cwlogdir = /opt/oscam/etc/cw
    disablelog = 0
    disableuserfile = 0
    usrfileflag = 0
    clienttimeout = 2000
    fallbacktimeout = 1500
    clientmaxidle = 120
    failbantime = 0
    bindwait = 120
    netprio = 0
    clientdyndns = 0
    resolvedelay = 2
    unlockparental = 0
    nice = -1
    serialreadertimeout = 1000
    maxlogsize = 10
    waitforcards = 1
    preferlocalcards = 1
    saveinithistory = 0
    readerrestartseconds = 5
    lb_mode = 0
    lb_save = 0
    lb_nbest_readers = 1
    lb_nfb_readers = 1
    lb_min_ecmcount = 5
    lb_max_ecmcount = 500
    lb_reopen_seconds = 900
    resolvegethostbyname = 0

    [newcamd]
    #port = 15050@0604:000000
    port = 15050@0B00:000000

    key = 0102030405060708091011121314
    allowed =
    keepalive = 1
    mgclient = 0

    [webif]
    httpport = 15080
    httpuser = username
    httppwd = password
    httprefresh = 0
    httpallowed = 127.0.0.1,192.168.0.107
    httphideidleclients = 0
    httpreadonly = 0

    oscam.server

    # reader configuration

    [reader]
    label =r eader1
    protocol = mouse
    detect = CD
    device = /dev/ttyUSB0
    group =
    emmcache = 1,3,2
    services = services1
    caid = 0B00
    mhz = 500
    cardmhz = 500

    oscam.user:

    [account]
    user = tvheadend
    pwd = tvheadend
    uniq = 0
    group = 1
    ident = 0B00:0
    caid = 0B00
    au = Digitenne

    Troubleshooting

    Q: The cardreader changes node from /dev/USB0 to /dev/USB1
    A: Unplugging and replugging the USB-cardreader might place the reader at another port, for example ttyUSB1, which will make OScam unable to find it.
    However, after reboot the cardreader should be back at /dev/ttyUSB0, so for a ‘working’ system where the device is not unplugged there should be no such problems.

    Q: The cardreader is not visible in /dev !
    A: You have to select support for this device in the kernel. In my case for the Smargo Smart Reader the driver is “USB FTDI Single Port Serial Driver”

  • MythTV: Firefox as default browser

    Using Firefox (Iceweasel) as the default browser for MythTV

    This page describes the configuration to set up Firefox as the default browser of MythTV, replacing the ‘ internal’ browser.
    Just changing the browser was the easy part, however I had a bit more trouble setting up the remote control for Firefox. In the end, (assuming Lirc works for Mythtv) setting up the remote to work with Firefox in Mythtv is not so hard, but I had to use some small tricks to get it working properly, and there was no step-by-step guide to get it working, so I decided to write down this small ‘howto’.

    There are two parts of this setup:
    Preparation: Starting Firefox from a shellscript.
    Create simple startup-scripts that will be used later on.

    A. Seting up Remote Control for Firefox (Iceweasel) using LIRC
    This allows you to start and control Firefox using a remote control.

    B. Using Firefox with remote control as the browser for MythTV
    This will make firefox the default browser for Mythtv.

    Prerequisite:
    The assumption is that you have lircd and mythtv installed and running.

    Preparation: Starting Firefox from a shellscript.
    We will use a shell-script to start Firefox since we want to execute some specific commands when Firefox is started from within Mythtv. Use nano/vi/anyeditor to create a file called firefox-start.sh, and put the following into the file:

    #!/bin/sh
    /usr/bin/firefox $1 --fullscreen &
    exit 0

    Then create a file firefox-stop.sh. killall is used since Iceweasel keeps running after returning to mythtv:
    sudo killall firefox-bin &

    Use:
    ~$ chmod 777 firefox-start.sh
    ~$ chmod 777 firefox-stop.sh

    to make the file executable.
    Test the script by issuing the following command from the prompt in the directory where the scipt is placed:
    ~$ ./firefox-start.sh

    This should start Firefox. If this works we are no ready to configure .lirc to allow execution ofthis script using keypresses on the remote control.

    A. Seting up Remote Control for Firefox (Iceweasel) using LIRC

    You can use the scripts to start firefox we have to define Remote Control and LIRC for firefox.

    Remote Control and LIRC for Firefox
    To start programs using the RC you can use ‘irexec’, and to send commands to a program use ‘irxevent’.
    We need to configure both, using IREXEC to start/stop Firefox, and using IRXEVENT to send commands to the browser once it is running.
    Both irexec and irxevent look at your .lirc file for key-bindings/definitions, so we first have to tell LIRC what to do when a button is pressed by adding the following to the .lirc file in your home directory ( in this example : /home/mediauser/.lirc ).

    begin
    prog = irexec
    button = Blue
    config = /home/mediauser/firefox-start.sh
    end

    begin
    prog = irexec
    button = Back
    repeat = 3
    config = /home/mediauser/firefox-stop.sh
    end

    begin
    prog = irxevent
    button = ChanUp
    repeat = 3
    config = Key ctrl-plus CurrentWindow
    end

    begin
    prog = irxevent
    button = ChanDown
    repeat = 3
    config = Key ctrl-minus CurrentWindow
    end

    This programs four buttons on your remote, more buttons can be added later:
    - the blue button starts ‘Firefox-start.sh’
    - the ‘back’ button starts ‘Firefox-stop.sh’
    - Channel up is equivalent to ctrl-plus, zooming in
    - Channel down is equivalent to ctrl-minus, zooming out

    Note that the first two are buttons for irexec, since they are used to start and stop the application, and the last two are buttons for irxevent since they represent buttons to be send to the running program in focus.

    Execute IREXEC and IRXEVENT in daemon-mode, replacing /home/mediauser with the path to your .lircrc-file:
    ~$ irexec -d /home/mediauser/.lircrc
    ~$ irxevent -d /home/mediauser/.lircrc

    This should do the trick; if no errors occurred both irexec and irxevent are listening.
    To test: Press the blue button on the remote, and Firefox should start. Press ‘back’ and Firefox should stop.

    Troubleshooting: What if this doesn’t work:
    - First make sure lircd is already running
    - Sometimes old instances of irexec or irxevents are still running, and the remote is still configured using the old mappings: Use
    killall irexec
    killall irxevent

    to stop them, and verify using ps -A to make sure they are all killed.Then try again to execute irexec and irxevent.

    B. Using Firefox with remote control as the browser for MythTV

    Using Firefox as your browser in Mythtv
    To change the browser, open Mythtv and go to ‘ setup’ , ‘ web’. There you find the command that starts the browser. The default is ‘internal’. You can change this to ‘/usr/bin/firefox %URL%’ to start firefox. However, we will use the shellscript we created earlier on since we want to add some specific command when firefox is started.

    /home/mediauser/firefox-start.sh %URL%

    The %URL%-part is needed to add the page mythtv wants to open to the command being executed.

    Switching on and off irexec when using Firefox

    Irxevent sends commands (like ‘up’ and ‘ down’) to any program in the active window. Somehow, when activated, it adds up to the commands already received by Mythtv from the .lirc-part of the file configered for mythtv. (that is, the part with prog=mythtv sends the command once, and the part with prog=irxevent sends the command another time). Consequence is that pressing ‘up’ once becomes equivalent to two times ‘arrow up’, which is unwanted (you would skip every other menu option :-) ). This can be solved by starting irexec and irxevents just before calling firefox, and by stopping irexec and irxevents just after stopping firefox.
    This can be achieved by adding the corresponding commands to the shell scripts for stopping and starting firefox.
    firefox-start.sh becomes:

    irexec -d /home/mediauser/.lircrc
    irxevent -d /home/mediauser/.lircrc

    #!/bin/sh
    /usr/bin/firefox $1 --fullscreen &
    exit 0

    firefox-stop.sh becomes:

    killall firefox-bin &
    killall irexec
    killall irxevent

    Contact
    If you have any remarks, questions or suggestion please send me an e-mail at: info[at]lex-web.net

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>