python 3, headless RaspPi, locale de_DE not possible for python-evdev

Question

After the main problem is solved try to read from multiple HID inputs I open here a new question for the following (and different) problem:

I installed minibian on a RaspPi. The entire system is running on 'de_DE', in details:

root@ddpi:~# locale
LANG=de_DE.UTF-8
LANGUAGE=
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=

and

root@ddpi:~# grep -v ^# /etc/locale.gen
de_DE.UTF-8 UTF-8

and also python locale.getdefaultlocale() says 'de_DE','utf-8'.

There is no X running (because it's not installed): ps -aux | grep X says 1297 0.0 0.1 4212 1828 pts/0 S+ 10:13 0:00 grep X and systemctl get-default says multi-user.target.

But there is the following problem: I have a German keyboard connected, which writes german as expected in console, but is recognized as english from evdev. I am using these scripts, read input from usb keyboard in linux in combination with reading events from multiple devices which works as expected, except the language. My Y keystroke is recognized as KEY_Z, my Z keystroke as KEY_Y, my ß as KEY_MINUS and so on. But everything, so far I can see, is set to de_DE.

This script (the evdev part only)

from evdev import InputDevice, categorize, ecodes
dev = InputDevice('/dev/input/event0')

print(dev)
#device /dev/input/event1, name "Dell Dell USB Keyboard", phys "usb-0000:00:12.1-2/input0"

for event in dev.read_loop():
    if event.type == ecodes.EV_KEY:
        print(categorize(event))

says

key event at 1510568613.966220, 44 (KEY_Z), down
key event at 1510568614.039341, 44 (KEY_Z), up
key event at 1510568614.926334, 21 (KEY_Y), down
key event at 1510568614.998331, 21 (KEY_Y), up
key event at 1510568615.799577, 12 (KEY_MINUS), down
key event at 1510568615.863569, 12 (KEY_MINUS), up

for yzß keystrokes.

What's wrong here ?

Hardwood

Hello @ddlab. This has to do with scancodes and keycodes (to clarify, if you had an AZERTY keyboard, pressing A would result in scancode KEY_Q and keycode a). The mapping of scancodes to keycodes is done by the virtual console driver. Evdev can only do scancodes. Mapping between the two is not straightforward, but I've been thinking of adding an example to python-evdev, since it's an often requested feature (I'll try this weekend). In the meantime, you can have a look at the keyboard project. – Amaze

@Amaze Thanks for your efforts. In my case the easiest rough and dirty way could be to expand the mapping by a language variable like this: findig language LANG = locale.getdefaultlocale()[0].split('_')[0].upper() and the map like CODE_MAP_CHAR = { 'DE' : { 'KEY_Z': "Y", 'KEY_Y': "Z" }, 'EN' : { 'KEY_Z': "Z", 'KEY_Y': "Y" } } and the function like this def parse_key_to_char(val): return CODE_MAP_CHAR[LANG][val] if val in CODE_MAP_CHAR[LANG] else "" – Hardwood

Answer 1

So. According my own suggestion to create a german keyboard mapping here are my two scripts, which are still DEV VERSIONS, not yet in production, but work correctly on console output. The first one I named hidraw_survey.py because it is wondering all the time, if new HIDs are connected or unplugged. Both events cause a pkill -f and restart of the second script read_hidraw_scanner.py, which handles the keyboard events of all connected HIDs at once. As there are the following:

device /dev/input/event4, name "OPTO-E Barcode Device", phys "usb-3f980000.usb-1.5.4/input0"
device /dev/input/event0, name "CHICONY HP Basic USB Keyboard", phys "usb-3f980000.usb-1.5.1/input0"
device /dev/input/event1, name "Sycreader USB Reader", phys "usb-3f980000.usb-1.5.2/input0"
device /dev/input/event2, name "HID 04d9:1203", phys "usb-3f980000.usb-1.5.3/input0"
device /dev/input/event3, name "HID 04d9:1203", phys "usb-3f980000.usb-1.5.3/input1"

Why the numerical keypad (last on list) is counted twice... no idea, but the input events are processed correctly.

Both scripts try to find out and guess the numbers of the input event connections between 0 and 9 and skip these numbers, on which the tests are not successful. The language specific mapping itself is realized by LANG = locale.getdefaultlocale()[0].split('_')[0].upper() which delivers me DE.

The first script hidraw_survey.py:

import sys
import os
from pathlib import Path
import time


logfile = '/ddpos/log/hidraw_scanner.txt'

f = open(logfile, 'a')
f.write('hidraw_scanner started: '+time.strftime("%d.%m.%Y %H:%M:%S")+'\n')
f.close()

def try_kill_script(script):
    try:
        os.system('pkill -f '+script)
    except OSError:
        return False
    else:
        return True

old = ''
new = ''
while 1:
    out = ''
    for i in range(10):
        hidrawfile = '/dev/hidraw'+str(i)
        hidrawpath = Path(hidrawfile)
        if hidrawpath.exists():
            out += str(i)
            #print (out)
    new = out
    if new != old:
        print ('change registered')
        f = open(logfile, 'a')
        f.write('USB changed to: '+out+'\n')
        f.close()
        try_kill_script('read_hidraw_scanner.py')
        os.system('python3 /ddpos/test/read_hidraw_scanner.py &')
    old = out

The 2nd script read_hidraw_scanner.py:

import os
import sys
from evdev import InputDevice, list_devices, ecodes, categorize
from select import select
from pathlib import Path
import locale

LANG = locale.getdefaultlocale()[0].split('_')[0].upper()

CODE_MAP_CHAR = {
    'DE': {
        'KEY_A': "a",
        'KEY_B': "b",
        'KEY_C': "c",
        'KEY_D': "d",
        'KEY_E': "e",
        'KEY_F': "f",
        'KEY_G': "g",
        'KEY_H': "h",
        'KEY_I': "i",
        'KEY_J': "j",
        'KEY_K': "k",
        'KEY_L': "l",
        'KEY_M': "m",
        'KEY_N': "n",
        'KEY_O': "o",
        'KEY_P': "p",
        'KEY_Q': "q",
        'KEY_R': "r",
        'KEY_S': "s",
        'KEY_T': "t",
        'KEY_U': "u",
        'KEY_V': "v",
        'KEY_W': "w",
        'KEY_X': "x",
        'KEY_Y': "z",
        'KEY_Z': "y",
        'SHIFT_KEY_A': "A",
        'SHIFT_KEY_B': "B",
        'SHIFT_KEY_C': "C",
        'SHIFT_KEY_D': "D",
        'SHIFT_KEY_E': "E",
        'SHIFT_KEY_F': "F",
        'SHIFT_KEY_G': "G",
        'SHIFT_KEY_H': "H",
        'SHIFT_KEY_I': "I",
        'SHIFT_KEY_J': "J",
        'SHIFT_KEY_K': "K",
        'SHIFT_KEY_L': "L",
        'SHIFT_KEY_M': "M",
        'SHIFT_KEY_N': "N",
        'SHIFT_KEY_O': "O",
        'SHIFT_KEY_P': "P",
        'SHIFT_KEY_Q': "Q",
        'SHIFT_KEY_R': "R",
        'SHIFT_KEY_S': "S",
        'SHIFT_KEY_T': "T",
        'SHIFT_KEY_U': "U",
        'SHIFT_KEY_V': "V",
        'SHIFT_KEY_W': "W",
        'SHIFT_KEY_X': "X",
        'SHIFT_KEY_Y': "Z",
        'SHIFT_KEY_Z': "Y",

        'SHIFT_KEY_GRAVE':'°',
        'SHIFT_KEY_1': "!",
        'SHIFT_KEY_2': "\"",
        'SHIFT_KEY_3': "§",
        'SHIFT_KEY_4': "$",
        'SHIFT_KEY_5': "%",
        'SHIFT_KEY_6': "&",
        'SHIFT_KEY_7': "/",
        'SHIFT_KEY_8': "(",
        'SHIFT_KEY_9': ")",
        'SHIFT_KEY_0': "=",
        'SHIFT_KEY_EQUAL': "`",
        'SHIFT_KEY_RIGHTBRACE':'*',
        'SHIFT_KEY_BACKSLASH':'\'',
        'SHIFT_KEY_COMMA':';',
        'SHIFT_KEY_DOT':':',
        'SHIFT_KEY_SLASH':'_',

        'ALTGR_KEY_Q': "@",
        'ALTGR_KEY_MINUS': "\\",
        'ALTGR_KEY_7': "{",
        'ALTGR_KEY_0': "}",
        'ALTGR_KEY_8': "[",
        'ALTGR_KEY_9': "]",
        'ALTGR_KEY_RIGHTBRACE':'~',

        'KEY_NUMERIC_STAR': "*",

        'KEY_GRAVE':'^',
        'KEY_LEFTBRACE':'Ü',
        'KEY_RIGHTBRACE':'+',
        'KEY_SEMICOLON':'Ö',
        'KEY_APOSTROPHE':'Ä',
        'KEY_BACKSLASH':'#',
        'KEY_102ND':'<',
        'KEY_COMMA':',',
        'KEY_DOT':'.',
        'KEY_SLASH':'-',
        'KEY_KPSLASH':'/',
        'KEY_KPASTERISK':'*',
        'KEY_KPMINUS':'-',
        'KEY_KPPLUS':'+',
        'KEY_KPDOT':'.',
        'KEY_SPACE': " ",
        'KEY_EQUAL': "´",
        'KEY_TAB': "\t",

        'KEY_MINUS': "ß",
        'KEY_SPACE': " ",

        'KEY_NUMERIC_1': "1",
        'KEY_NUMERIC_2': "2",
        'KEY_NUMERIC_3': "3",
        'KEY_NUMERIC_4': "4",
        'KEY_NUMERIC_5': "5",
        'KEY_NUMERIC_6': "6",
        'KEY_NUMERIC_7': "7",
        'KEY_NUMERIC_8': "8",
        'KEY_NUMERIC_9': "9",
        'KEY_NUMERIC_0': "0",
        'KEY_KP1': "1",
        'KEY_KP2': "2",
        'KEY_KP3': "3",
        'KEY_KP4': "4",
        'KEY_KP5': "5",
        'KEY_KP6': "6",
        'KEY_KP7': "7",
        'KEY_KP8': "8",
        'KEY_KP9': "9",
        'KEY_KP0': "0",
        'KEY_1': "1",
        'KEY_2': "2",
        'KEY_3': "3",
        'KEY_4': "4",
        'KEY_5': "5",
        'KEY_6': "6",
        'KEY_7': "7",
        'KEY_8': "8",
        'KEY_9': "9",
        'KEY_0': "0"
    },
     'EN': {
        'KEY_A': "A",
        'KEY_B': "B",
        'KEY_C': "C",
        'KEY_D': "D",
        'KEY_E': "E",
        'KEY_F': "F",
        'KEY_G': "G",
        'KEY_H': "H",
        'KEY_I': "I",
        'KEY_J': "J",
        'KEY_K': "K",
        'KEY_L': "L",
        'KEY_M': "M",
        'KEY_N': "N",
        'KEY_O': "O",
        'KEY_P': "P",
        'KEY_Q': "Q",
        'KEY_R': "R",
        'KEY_S': "S",
        'KEY_T': "T",
        'KEY_U': "U",
        'KEY_V': "V",
        'KEY_W': "W",
        'KEY_X': "X",
        'KEY_Y': "Y",
        'KEY_Z': "Z",
        'KEY_GRAVE':'`',
        'KEY_MINUS': "-",
        'KEY_SPACE': " ",  
        'KEY_BACKSLASH': "\\",
        'KEY_GRAVE': "`",
        'KEY_NUMERIC_STAR': "*",
        'KEY_LEFTBRACE': "[",
        'KEY_RIGHTBRACE': "]",    
        'KEY_COMMA': ",",
        'KEY_EQUAL': "=",    
        'KEY_SEMICOLON': ";",
        'KEY_APOSTROPHE': "'",
        'KEY_TAB': "\t",
        'KEY_DOT': ".",
        'KEY_SLASH': "/",
        'KEY_NUMERIC_1': "1",
        'KEY_NUMERIC_2': "2",
        'KEY_NUMERIC_3': "3",
        'KEY_NUMERIC_4': "4",
        'KEY_NUMERIC_5': "5",
        'KEY_NUMERIC_6': "6",
        'KEY_NUMERIC_7': "7",
        'KEY_NUMERIC_8': "8",
        'KEY_NUMERIC_9': "9",
        'KEY_NUMERIC_0': "0",
        'KEY_KP1': "1",
        'KEY_KP2': "2",
        'KEY_KP3': "3",
        'KEY_KP4': "4",
        'KEY_KP5': "5",
        'KEY_KP6': "6",
        'KEY_KP7': "7",
        'KEY_KP8': "8",
        'KEY_KP9': "9",
        'KEY_KP0': "0",
        'KEY_1': "1",
        'KEY_2': "2",
        'KEY_3': "3",
        'KEY_4': "4",
        'KEY_5': "5",
        'KEY_6': "6",
        'KEY_7': "7",
        'KEY_8': "8",
        'KEY_9': "9",
        'KEY_0': "0"
    }
}

def parse_key_to_char(val):
    return CODE_MAP_CHAR[LANG][val] if val in CODE_MAP_CHAR[LANG] else ""

def try_kill_script(script):
    try:
        os.system('pkill -f '+script)
    except OSError:
        return False
    else:
        return True

founddev = []
for i in range(10):
    trydev = '/dev/input/event'+str(i)
    if Path(trydev).exists():
        founddev.append(trydev)

devices = map(InputDevice, founddev)
devices = {dev.fd: dev for dev in devices}

for dev in devices.values():
    print(dev)

chars = ''
prefix = ''

while 1:

    r, w, x = select(devices, [], [])
    for fd in r:
        try:
            for event in devices[fd].read():
                if event.type == ecodes.EV_KEY:
                    e = categorize(event)
                    if e.keystate == e.key_down:
                        if e.keycode == 'KEY_LEFTSHIFT' or e.keycode == 'KEY_RIGHTSHIFT':
                            prefix = 'SHIFT_'
                        elif e.keycode == 'KEY_LEFTALT':
                            prefix = 'ALT_'
                        elif e.keycode == 'KEY_RIGHTALT':
                            prefix = 'ALTGR_'
                            print ('prefix='+prefix)
                        elif e.keycode == 'KEY_ESC':
                            chars = ''
                        elif e.keycode == 'KEY_BACKSPACE':
                            chars = chars[:-1]
                        elif e.keycode == 'KEY_ENTER' or e.keycode == 'KEY_KPENTER':
                            sys.stdout.write(chars+'\n')
                            sys.stdout.flush()
                            chars = ''
                        else:
                            #print (ecodes.EV_LED) # which outputs 17 forever
                            print (prefix + e.keycode)
                            chars += parse_key_to_char(prefix + e.keycode)
                    elif e.keystate == e.key_up:
                        #sys.stdout.write(parse_key_to_char(e.keycode))
                        #sys.stdout.flush()
                        #print ("'"+e.keycode+"':'"+parse_key_to_char(e.keycode)+"',")
                        if e.keycode == 'KEY_LEFTSHIFT' or e.keycode == 'KEY_RIGHTSHIFT':
                            prefix = ''
                        elif e.keycode == 'KEY_LEFTALT':
                            prefix = ''
                        elif e.keycode == 'KEY_RIGHTALT':
                            prefix = ''
        except:
            try_kill_script('read_hidraw_scanner.py')

Both scripts are full of dents and wrinkles made by a python newbie like me, but I think with a bit of improvement and simplifications it's a really nice toy to tell the RaspPi something like config settings or password inputs or whatever (in my case a barcode scanner with alternative manual input in case of not readable barcode and a RFID reader for waiter/waitress authentification in restaurants)

The mapping covers all german keyboard keys including SHIFT and ALTGR (right ALT), not yet ALT (left ALT). But it doesn't work with accents like è or é or ñ, nor CAPSLOCK. Improvements are welcome.

UPDATE The output looks like that at the moment. One after the other, a scanned numeric barcode, an alphanumeric barcode, a keyboard input and the RDFID reader at last.

KEY_0
KEY_4
KEY_0
KEY_1
KEY_3
KEY_6
KEY_0
KEY_0
KEY_0
KEY_0
KEY_4
KEY_8
KEY_6
KEY_0
KEY_9
KEY_5
0401360000486095
KEY_2
SHIFT_KEY_B
KEY_5
KEY_6
KEY_6
KEY_9
SHIFT_KEY_C
SHIFT_KEY_F
KEY_4
SHIFT_KEY_A
KEY_2
KEY_1
SHIFT_KEY_C
KEY_1
SHIFT_KEY_B
KEY_7
2B5669CF4A21C1B7
KEY_M
KEY_Z
KEY_N
KEY_A
KEY_M
KEY_E
ALTGR_KEY_Q
KEY_E
KEY_M
KEY_A
KEY_I
KEY_L
KEY_DOT
KEY_C
KEY_O
KEY_M
[email protected]
KEY_0
KEY_0
KEY_0
KEY_5
KEY_6
KEY_1
KEY_1
KEY_4
KEY_9
KEY_6
0005611496

Hardwood