Category: software Page 1 of 2

Undelete Files with SnapRAID

Today I accidentally deleted a whole folder of important documents on my OpenMediaVault(OMV) NAS, while doing some cleaning up.
Of course I would have a backup of the folder on an external hard-drive, but this was the perfect opportunity to test SnapRAID.

SnapRAID is a software RAID which comes as a plugin for OMV. SnapRAID computes redundancy/parity information for the files stored on multiple data disks. This redundancy/parity must be stored on a separate disk. If one data disk fails, the lost data on that disk can be reconstructed from the redundancy/parity information and the other remaining data disks. SnapRAID is quite flexible and allows any number of data disks and redundancy disks and a lot of options, which I haven’t even looked at to be honest.
The most important difference to other RAID systems is that SnapRAID is an offline RAID. This means SnapRAID will not constantly monitor disks and update the redundancy in real-time. Instead you are required to run SnapRAID manually every couple of days/weeks or schedule a cronjob to periodically update the SnapRAID parity disk.

Enough of that, here is how a deleted file or folder can be restored. The SnapRAID plugin for OMV allows to do this from the web UI, but I ended up using the shell anyway.

To check which files have been removed (or added) since the last parity update:
$> snapraid diff --test-fmt path
or
$> snapraid diff --test-fmt disk

To fix an entire array, i.e. fix everything that is missing:
$> snapraid fix -m
To fix everything on one disk (example disk name “MyDisk2” as assigned during SnapRAID creation):
$> snapraid fix -m -d MyDisk2
To fix a folder with a given name (located on any data disk):
$> snapraid fix -m -f myFolder/
or a file:
$> snapraid fix -m -f myFile
To just check what SnapRAID would do instead of fixing errors right away do:
$> snapraid check -m -f MyFile -v

After a couple of minutes my folder was back where it belonged. Feels good to be prepared for mishaps like that 🙂

References:
https://www.snapraid.it/manual
https://sourceforge.net/p/snapraid/discussion/1677233/thread/a7db07e5/

Getting Started with Magic VLSI

Magic VLSI – or just Magic – is a free and open source VLSI layout software. Simply put Magic allows you to draw the mask layers used in a semiconductor facrication process. The Magic software is another “Berkeley Child” (like BSD and others) and first came into existence in the 1980s. Magic is still under active development as of late 2019.

Some Linux distributions offer a pre-build package for Magic from their package repository. Most often these packages are outdated and therefor it is best to build Magic from the sources.

Start by pulling the latest release from the official project page.
$> git clone git://opencircuitdesign.com/magic
$> cd magic

$> git checkout magic-8.2

I am building Magic in a WSL Ubuntu 18.04 while writing this guide, so installing some additional packages is required before continuing. This step is subject to the build machine and may vary.
$> sudo apt install csh
$> sudo apt install libglu1-mesa-dev freeglut3-dev mesa-common-dev
$> sudo apt install tk tk-dev
$> sudo apt install libcairo2 libcairo2-dev

Now let’s continue to build Magic. I like to install software to /opt so it’s not burried deep inside the file tree and is available to all users on my machine (which is just me). The installation folder is set by the --prefix option.
$> ./configure --prefix=/opt/magic-8.2
If all the dependencies above where installed you should see a fivefold yes after the configure script has completed.

With that out of the way the build process can be launched as usual. The installation step may require to change the permissions on the installation folder, so that the current user has write access to the installation path.
$> make
$> make install

That’s it! Let’s see if it worked.
$> cd /opt/magic-8.2/bin
$> ./magic &

Success! Magic 8.2 has been launched.

In order to have something more than a gray box to look at let’s load the layout of an example gate from my Magic VLSI examples repo.

“Nice colors!” “Oh NAND you.”

That’s it for now. More examples on how to get started with Magic may follow.
P.S.: Before I forget to mention it, Magic requires a mouse with 3 buttons to work as intended. No kidding!

References:
[1] http://opencircuitdesign.com/magic/
[2] http://www.codebind.com/linux-tutorials/install-opengl-ubuntu-linux/
[3] https://github.com/andrsmllr/magic_vlsi_examples

Getting Started with GHDL

If you haven’t heard of GHDL, it is *the* free open-source VHDL simulator out there.
GHDL stand for “G Hardware Description Language” (the G is without meaning). GHDL is mainly implemented in Ada and can be build with different backends: mcode, LLVM and GCC. The different backends provide different performance levels and vary in build complexity. I recommend LLVM since it performs well and is still quite straight forward to build. Building GHDL from latest sources from its github project is probably the best way to go.

Despite its free nature GHDL provides very good support for all major VHDL-LRM releases: VHDL-1987/1993/200X/2008(partial). Unforunately GHDL is a pure VHDL simulator, so there is no support for Verilog at all. This is understandable as there are already some very good simulators for Verilog out there.

Compiling GHDL

The following guide assumes a Ubuntu 18.04 environment (either a native installation or docker or WSL will do).
Clone the latest GHDL sources or any stable release from github:
$> git clone https://github.com/ghdl/ghdl
Decent into the ghdl working copy and run the configure script with options to use LLVM as backend (–with-llvm-config) and a custom install path (–prefix):
$> cd ghdl
$> ./configure --prefix=/opt/ghdl-llvm --with-llvm-config

Install some dependencies:
$> sudo apt install -y bison flex
Afterwards you can build and install ghdl:
$> make
$> make install

The ghdl main executable is located at /opt/ghdl-llvm/bin/ghdl (the path given to the –prefix option). I usually create a symbolic link to make the ghdl command directly available in the $PATH:
$> ln -s /opt/ghdl-llvm/bin/ghdl /usr/bin/ghdl

That’s it for now. If you are familiar with docker, there is an easy to use docker image for ghdl available on dockerhub.

Anlogic TANG PriMER dev board

Recently I purchased a Sipeed TANG PriMER development board featuring an Anlogic EG4S20 FPGA (codenamed Eagle S20). The only reason I bought the board was to see what Anlogic FPGAs are capable of, since I had never heard of that FPGA vendor before. No need to think twice when the board costs less than 20$.

The TANG PriMER board is officially marketed as a RISC-V development board and comes with a Hummingbird E200 RISC-V softcore design preloaded into the onboard configuration flash. The Hummingbird is basically a slightly modified variation of the SiFive E2 core.
Setting up the tool chain was a bit of a hustle until I found this site which hosts both the TD IDE and the required license files. There are also some datasheets and schematics. Most of the official documentation is only available in Chinese, therefor I strongly recommend the inofficial english translation.

I have not done a lot with this board yet but verify the tool chain with a simple blink LED example. The design included a 32 bit counter which resulted in an estimated maximum frequency of 252 MHz. Not too bad. The TD IDE also comes with an IP wizard to generate IP cores, but it seems to just generate a wrapper for some primitive instantiations. It’s worth mentioning that the EG4S20 has an on-chip oscillator (250 or 266 MHz, documentation and IDE do not agree), on-chip SDRAM (64Mbit) and an 8-channel ADC (1MHz sample rate). Still need to figure out how to configure the board and which programmer can be used.

Since Anlogic FPGAs are not listed on digikey.com or other distributor websites, it seems unlikely they will become widely available outside of China anytime soon.

WSL on Windows 10

There is a new kid in town when it comes to “doing Linux under Windows”. Up to now there was primarily Cygwin and MinGW. But for a while now Microsoft offers the Windows Subsystem for Linux (WSL). I personally think the name should be the other way around (LSW) but what do I know.

To enable WSL under Windows 10 you first need to enable the required feature.
1. From the start menu go to “Apps and Features”
2. Scroll to the bottom and select “Programs and Features”
3. Click “Turn Windows features on or off” on the left side.
4. Set the check mark on “Windows Subsystem for Linux”
5. Finally a reboot is required for the changes to take effect.

That’s it. Almost. Now we need to install a Linux distribution to run. WSL only brings the environment for Linux to “run on” Windows 10, but there is no default distro included.
The easiest way is to go the App Store and install one of the available distros, like Ubuntu 18.04.
Alternatively you can download the required .appx package and install it manually using the PowerShell (Administrator permissions required) with
Add-AppPackage <filename&gt;.appx

Let’s see when Windows will completely adopt the Linux Kernel.

How to set up NordVPN on Ubuntu

There are two ways to set up a VPN connection with NordVPN on Ubuntu. The first one is to add NordVPN’s package repository to the apt sources and then install the nordvpn client via a simple sudo apt install nordvpn. The second one is to install OpenVPN and configure it to use the servers of NordVPN.
This how-to was written for Ubuntu 18.04 LTS (Bionic Beaver).

NordVPN setup with NordVPN package repository

  1. Download the NordVPN package (.deb) for Ubuntu:
    wget https://repo.nordvpn.com/deb/nordvpn/debian/pool/main/nordvpn-release_1.0.0_all.deb
  2. Install the package on your Ubuntu system:
    sudo dpkg --install ./nordvpn-release_1.0.0_all.deb
    Note that installing this package does only add the NordVPN package repository to apt, no executable is installed.
  3. Install the nordvpn client software:
    sudo apt update
    sudo apt install nordvpn
  4. Connect the nordvpn client (the server is selected automatically):
    nordvpn connect
    The general command to connect has the form:
    nordvpn connect [<country&gt;] [<city&gt;] | [server]
    so to connect to a specific country (and city) the additional arguments must be given:
    nordvpn connect United_Kingdom Manchester
    To get a list of available countries and cities you can query them with:
    nordvpn countries
    nordvpn cities
    It is also possible to give the server name directly:
    nordvpn de442

NordVPN setup with OpenVPN

  1. Install openvpn on your Ubuntu system:
    sudo apt install openvpn
  2. Get the OpenVPN configuration files for NordVPN:
    cd /etc/openvpn
    sudo wget https://downloads.nordcdn.com/configs/archives/servers/ovpn.zip
    sudo unzip ./ovpn.zip
    If an error is encountered during the download step above, it may be necessary to install the ca-certificate package:
    sudo apt-get install ca-certificates
    After unzipping ovpn.zip there will be a folder ovpn_tcp and ovpn_udp which contain the NordVPN server configuration files for OpenVPN.
  3. Connect to OpenVPN using one of the NordVPN servers:
    openvpn /etc/openvpn/ovpn_udp/de442.nordvpn.com.udp.ovpn
    You will need to enter your credentials for NordVPN.
Sources:
https://nordvpn.com/de/download/linux/
https://nordvpn.com/de/tutorials/linux/openvpn/
https://wiki.ubuntuusers.de/NetworkManager/VPN%20Plugins/

Realizing Arbitrary Functions with ROM-Based Lookup Tables

When tasked with the implementation of a rather complex function, e.g. a polynomial of higher order, the resource utilization quickly shoots through the roof if implemented straight forward (also called the naïve implementation).
To avoid this it is often easier, simpler and faster to use a lookup table (LUT) solution.

Instead of doing a lot of calculations and mathematics, the results for a given function argument is just read from a read-only memory (ROM) which contains precalculated results.
Oftentimes a LUT/ROM based implementation can be used in place of a “proper” implementation during early prototyping. In a later stage of the project the LUT/ROM can be replaced with an optimized implementation.

Arbitrary function realized using a LUT/ROM

Most people will encounter a ROM based lookup table solution when dealing with sine and cosine functions. The technique is then often called direct digital synthesis (DDS), because a waveform is generated by digital logic instead of analog circuitry, as it was done in ancient times.
Since this is a more or less common task I wrote an Octave script that takes a function as input and generates a memory initialization file for a LUT/ROM solution. The parameter range of interest can be specified and the width and depth of the LUT/ROM can be defined.
Periodic functions will likely result in problems if not handled carefully. E.g. a lot of people will specify the parameter range for a sine LUT/ROM go from 0 to 2*Pi. However, since the boundaries of the interval are always part of the LUT/ROM the start/end value of the period will appear twice, once at the highest LUT/ROM address and once at the lowest LUT/ROM address (because sin(0) = sin(2*Pi) = 0). This gotcha does not hold for non-periodic functions. It can also easily be fixed.

The quality of the result this technique yields depends on both the LUT/ROM’s memory depth and the LUT/ROM’s word width. The former defines the number of available sampling points and thus the quantization of the function parameter(s). The latter defines the quantization of the function result, which means how close a single LUT/ROM data value is to the exact result of the function. Both kinds of quantization contribute to the error of the LUT/ROM implementation, i.e. the deviation of the LUT/ROMs result from the precise function result value.

The total size of the LUT/ROM is memory depth * word width bits.
Another performance characteristic is the maximum operating frequency under which an implementation can run. On FPGAs the maximum operating frequency will depend on the number of BRAMs required to realize the LUT ROM. If only one BRAM is used there is not need for additional routing or coupling glue logic and thus the operating frequency will be maximized.
Using more than one BRAM requires “chaining”/”coupling” of BRAMs and thus will reduce the maximum operating frequency due to additional routing and/or logic delays.

LUT/ROM composed of multiple BRAMs

Extracting MPEG2-TS from Wireshark capture file

When working with IP multicast streams it can sometimes be useful to convert a captured IP stream file to a corresponding MPEG2 transport stream (TS) file. Wireshark has a built-in capability to extract MPEG2-TS packets from the UDP packets of an IP multicast stream. This feature is somewhat hidden away in the menus and not obvious to find.

An IP multicast stream

First select a frame of the UDP stream you are interested in by clicking on it. Then navigate the menu bar to “Analyze” &gt; “Follow” &gt; “UDP Stream”.

Follow the UDP stream

A new window will open, showing the contents of the UDP stream in one continuous flow of data. Now select “Show and save data as” &gt; “Raw” and hit “Save as …”.

Extract MPEG2-TS from UDP stream

Give the new file a name and hit “Save”.

It is also possible to filter out specific parts of the UDP stream, but in most cases the “Entire conversation” is what is wanted.

The result: a plain MPEG2-TS

That’s it! All MPEG2-TS packets have been extracted from the UDP stream and are stored to a separate file. The MPEG2-TS file can now be used for other tasks, like error checks or timing analysis.

STM32 Programming Entry

For the STM32 family of microcontrollers a number of different support libaries are available: Cortex Microcontroller Software Interface Standard (CMSIS), Standard Peripheral Library (SPL), Hardware Abstraction Layer (HAL).

CMSIS (by ARM) is basically just a bunch of header files with common defines for all registers of the microcontroller and its periperal devices. There are no API functions or drivers included at all. The CMSIS gives the programmer the most control and performance, but requires the most work.

SPL (by ST Microelectronics) consists of libraries allowing to use peripheral devices of the STM32 microcontroller much easier and abstracting away most of the wiggle work of register bits. The SPL is officially obsolete and will no longer be supported or maintained.

HAL (by ST Microelectronics) is the most recent library to support STM32 microcontrollers. It’s similar to SPL but offers some advanced features. Code written for HAL is supposed to be easily portable between different microcontrollers. On the other hand performance may not be at maximum since a lot of different hardware is supported by the same functions/API.

There is also a new thing called the Low Layer Library (LLL). The LLL defines different abstraction layers and allows the programmer to decide how much low level control she wants to take.

For more details check out the articles below.

References: https://blog.domski.pl/spl-vs-hal-which-one-should-you-use/

More Windows 10 Issues

On my new Windows 10 installation I came across a few more issues:

LibreOffice did have a nasty offset where the mouse pointer would click way above where it was pointing. The solutions to this bug was to disable OpenGL rendering as described here.

Another bad experience was with VirtualBox which would sometimes boot up virtual machines with corrupted graphics where nothing is readable or even recognizable. The solution to this behavior seems to be to set the “Override high DPI scaling behavior” setting to “Application”; the setting can be found by right clicking on the VirtualBox shortcut/executable &gt; Properties &gt; Compatibility &gt; Override high DPI scaling behavior. I found the solution here.

Page 1 of 2

© bananatronics.org