Thursday, March 5, 2015

Stream movies to Android devices



    After battling for some time with how to best stream movie content through Ethernet and Wifi from a media server (the Minix Neo Z64 running Linux in this case) where my USB hard disks are plugged, to Android devices connected to my TVs, I've found that the very best method for smooth streaming is to serve the files with a web server.


      That way, on any Android in the house (phone, tablet, TV box, MiniPC, or even an x86 PC) I can use a Web Browser to connect to the Z64 (with its static IP), then navigate my movie folder and click on the link of the movie I want to watch.
Android will prompt with what movie player (I use MX Player, and BSPlayer when MX has trouble) and playing will start, without installing any other app at all!

       I have even been able to have XBMC (now Kodi) on an Android device to read and manage this web-served movie collection just by adding it as an http server (however very long filenames are cut, making it harder for XBMC to link it to the Internet movie database).

For the basics of a NAS on Linux, you can read this article, which was for my original setup with the RK3188.


To serve the files just install on the Minix Neo Z64 a nice light web server, such as nginx. Then it's just a matter of editing the file /etc/nginx/sites-enabled/default to add a section for your file-web-server, like this:

server {
    charset utf-8;
   
    root /mnt;
    index index.html index.htm;

    server_name 192.168.1.4;

    location / {
        root /mnt/;
        try_files $uri $uri/ /index.html;
        autoindex on;
        autoindex_exact_size off;
        autoindex_localtime off;
    }
}

You'll just have to change the IP for the static one of your Z64, and the two lines that start with "root" which should point to the root folder that you want to share (subfolders will be shared too).

The "charset" line is very important for all of us with accented languages. The "autoindex" lines do the magic of creating a link for each file in a folder.


Then, on any device on your local network enter the Z64's IP on a web browser and you'll see the files and folders under /mnt in your Z64.


I usually install also the openssh-server so I can ssh into the Z64 from my PC, instead of connecting it to a TV.




BONUS: Add a script to init that executes one line like this for every hard disk (substitute the sd_) connected to the media server:
sudo hdparm -S 20 -B 50 -K 1 /dev/sd_
 This invaluable piece of advice makes sure that the hard drivers spin off when not used for some time (half an hour I think). This will enhance by orders of magnitude the life of your precious hard disks, and will reduce your energy bill.

Getting Linux on the Minix Neo Z64

      I have recently received the latest Minix Neo Z64, courtesy of Minix. This TV Box is equipped with an Intel x86 processor, an Atom 3735F, which is not so far off a low-end PC desktop but at just 2.2W making it fanless. You can check its respectable specs at this Intel page, and even compare it against other Intel CPUs.



The Z64 is smaller than previous Minix TV boxes, which is nice. As always, a good quality power adapter, HDMI cable, and remote are bundled in the box. The Wifi antenna design is non-detachable, and this time I have been unable to find a way to open the box in order to take the customary PCB photo. Sorry about that :-(

Apart from that, you get 32 GB fast eMMC flash (instead of the usual slower NAND flash chip), 2 GB DDR3 RAM at 1333 MHz, two (ouch) USB 2.0 ports, Ethernet 10/100 (ouch), Wifi and Bluetooth.

I find several big reasons to prefer this TV box over ARM ones:

- the x86 architecture is completely PC SW compatible (think Adobe Flash)

- Intel has a solid background on Open Source, like its Linux GPU drivers (2D/3D) which are top of the line. You even have 3D support on the HDMI interface (see 5.4.1.3 at the CPU datasheet here).

- For HW accelerated video decoding you have a VPU supporting 1080p60 decoding for H-264,  WMV9, and MPEG-2 (see 1.4 on Type 3 SoC at the CPU datasheet here).
 
- And one important thing: the USB controller. Usually ARM CPUs have DesignWare's USB controller, which has nasty and buggy firmware/driver, leading to lots of problems with USB devices (webcams, external hard disks, 10/100 USB Ethernet and Wifi dongles). However, here we have a good and robust Intel USB controller.

The last reason alone has made me jump into switching my NAS home media server (which has several USB hard disks) to this Minix Neo Z64 with Linux.

This tiny Intel CPU even has virtualization features (VT-x), you could even install Xen on this (or even Qubes-OS, if you are serious about secure computing).

So... sorry Windows 8.1, you look nice and you run fast, but Linux and I...


Hence, find here instructions to get into the Minix Neo Z64 whatever distribution of Linux you prefer. I don't like creating distros when you can very easily install a standard one.



Live Linux USB


     First of all get a USB pendrive that can be erased, so we can start Linux on the Z64 to make a backup of Windows, before removing it.

I've developed a liking for Xubuntu (XFCE based) after Canonical decided to outsmart all of us by inventing their hateable GUI, and then Gnome followed suit with its 3.0 version.

You can use any version of Ubuntu, they'll work the very same way as described here. Just head to the corresponding official Ubuntu page and download the x86_64 version (also called amd64) of the distro.

I wish I could tell you that's it, but truth is somebody messed up somewhere and all Intel Atom boxes/MiniPCs I know happen to have a 32 bits UEFI BIOS, which complicates booting a 64 bits OS, so there is one extra step.

This consists of adding a 32 bits bootloader to the 64 bits one in the Ubuntu distro so it can boot, that's it. You can either download the bootia32.efi bootloader from this link or you can follow the instructions (from the same place) to build it yourself.

Now, about creating a LiveUSB for UEFI systems and modifying it... people around are resorting to using a tool called Rufus that requires you to use Windows... ugly.

However, there is a simple way (thanks lopaka) to do everything from your usual Linux PC (the lines preceded with "#" are just my comments, disregard them):


#first erase the USB pendrive
sgdisk --zap-all /dev/sd_
#create a new GPT partition for booting on UEFI systems
sgdisk --new=1:0:0 --typecode=1:ef00 /dev/sd_

#create a FAT32 partition, as expected by the UEFI BIOS
mkfs.vfat -F32 /dev/sd_1

#create a temporary folder

mkdir image
#mount the FAT32 partition (only root can mount)
sudo mount -t vfat /dev/sd_1 image/

#uncompress Ubuntu onto the mounted USB
sudo 7z x xubuntu-14.10-desktop-amd64.iso -oimage/

#add the 32 bits bootloader (see blog above)
sudo cp bootia32.efi image/EFI/BOOT/

#finish any pending writes
sudo sync

#unmount the USB
sudo umount image

#remove the temporary folder
rmdir image

Please note that ALL references to the USB disk above are in the form "sd_" or "sd_1", this is because you can delete your hard disk if you just copy paste this code, so don't!!! You first have to find if your USB pendrive is at sdb, sdc, sdd, sde, or whatever sd*.
Easy way is, without the USB drive inserted, type in a terminal:
ls /dev/sd*
And then again, after inserting the USB. The new sd_ (without trailing number) is your pendrive. Use that letter instead of the "_" above.



BIOS settings


   In order to boot the Live USB, connect a keyboard to the Minix and as soon as you press the power button, start pressing several times the "Del" key to enter BIOS setup (if screen remains black after 10s, then you'll need a PC monitor instead of a TV to see it).

BTW, the sensible way to forcefully power off the box, if there is nothing on screen, is to keep pressed the power button for a bit more than 5 seconds.

On the BIOS screen, go to "Advanced", then "CPU Configuration" and on the "Execute Disable Bit" hit Enter and select Disabled.


By the way, just in case you dig through the BIOS options, you may see one named "ISP PCI Device Selection", just ignore it. I say this because it mentions that it should be another value for Linux, but never mind that, leave it at Windows.

However, there is a very important change to be done, find the "Quiet boot" item. It must be disabled since there seems to be something missing in the GFX initialization that the BIOS is doing but Linux is missing. Maybe the logo that appears (when Quiet boot is enabled) instead of the usual BIOS text is pushing the GPU into an incompatible video mode, and it prevents booting Linux (black screen).

Head to the last menu page (the one on Boot options) and select the option to Save Settings (without reset or exit, just save).
 
Finally, if the above instructions for the USB preparation were followed, you should see the pendrive's name or brand listed in the "Boot Override" choices at the bottom, select it and hit Enter to boot.

On the grub2 menu that should appear select "Try Ubuntu before installing".
If you get a black screen or Windows, then you may be missing the bootia32.efi file, re-do the instructions above on USB preparation.


Backup Windows 8.1


      Since Minix bundles a fully licensed Windows 8.1 with Bing (as opposed to many other TV boxes that come with a 30 days trial version), it is a good idea to back it up to avoid losing the licensed OS, in case we wanted to switch back to it anytime.

This part of the instructions is based on this CNX-Software's page, the news site for hacking with embedded devices; though there are additional things to make Linux work on the Z64.

We will install "pv" as a convenience, to monitor the copy's progress, since the backup takes several minutes to complete.

Just open a terminal and cd into the folder of another USB pendrive (the Live USB is read only so nothing survives reboot) to execute these commands:

sudo apt-get install pv
sudo dd if=/dev/mmcblk0 bs=16M | pv | gzip -9 > Z64_Win81.gz

Most of the 32 GB flash is empty, so gzip'ing it produces a much smaller file (mine was some 4 GB, instead of 32 GB)

To restore Windows 8.1 from the backup you should follow this section up to the "pv" install, and then do:

gzip -dc Z64_Win81.gz | pv | sudo dd of=/dev/mmcblk0 bs=16M


Installing Linux in Flash


Finally! Click on the icon to Install Ubuntu and start the setup as usual.

When asked about the partitioning select the do "Something else" choice and you'll be presented with the current partitioning of the internal eMMC flash that contains Windows 8.1.

We are going to wipe Windows. But you have to leave the first two partitions untouched, they are mmcblk0p1 and mmcblk0p2. Together they amount to less than 240MB and are needed to boot EFI (at least p1, the FAT32 one).

However, you can perfectly remove the "Win" and "Images" partitions, which together are around 30 GB of space.

I used all but 2 GB for an ext4 partition to be mounted at "/" where Linux will reside, and the remaining 2 GB for swap partition (the ancient recommendation is to match your RAM size in swap).

Even if this is a Flash, it's ok to have a swap partition, it should be seldom used, only when you start filling up the RAM, but a home server should be ok even without it.

On the booloader device dropdown at the bottom of the new window, select "/dev/mmcblk0" not other, just that one (or update-grub2 later may update the wrong place and you'll end up with two "ubuntu" entries on the UEFI BIOS).


Once installation is finished (this is based on Linuxium's great work here), you'll need to boot once more with the Live USB, but when presented with its grub2 menu, press 'c' to go to its console and enter these three lines:

linux (hd1,gpt3)/boot/vmlinuz-3.16.0-31-generic root=/dev/mmcblk0p3
initrd (hd1,gpt3)/boot/initrd-3.16.0-31-generic
boot






If you used a different Linux distro, the version numbers may be different. You can use grub's auto-completion, with the Tab key, to help you find your vmlinuz and initrd files for you.

 The Ubuntu that you have installed on the internal eMMC flash will boot up, connect to the Internet with an Ethernet cable (there is no wireless out of the box) and enter these commands on a terminal:

sudo apt-get install grub-efi-ia32 grub-efi-ia32-bin
sudo cp /boot/efi/EFI/ubuntu/grubia32.efi /boot/efi/EFI/ubuntu/grubx64.efi
sudo update-grub2


Now you can power off, remove the pendrive, and boot again

You should then boot to your Linux.


If you see a Windows bluish screen stating that there is something wrong, then you'll have to enter the BIOS once more and change the Boot priority #1 to "ubuntu".


Limitations


Please note Wifi (Realtek RGN RTL8723BS) is not working out of the box, but CNX-Software's page provides info on compiling its drivers from source.

Also the sound (Realtek ALC5642) is not working, there is one person around that says to have fixed it by using Chromium's Intel firmware (see this page) but I've tried it and, even though a new output appeared in the audio mixer, nothing sounded and the "dmesg" log was full of errors related to audio.

If you google around for ALC5642 and Linux there seems to be a good place to look for the driver (see this page) whenever it appears.


That's it. Enjoy!






Tuesday, June 10, 2014

Minix Neo X8H HW tinkering first impressions

I just got my hands on a Minix Neo X8-H TV Box, along with an M1 airmouse, courtesy of Minix, with the aim of porting Linux to it (Amlogic Quad-Core processor this time).


First impressions


I won't bother you with the specs, which you can review at Minix' X8-H product page. However, from a subjective point of view I can say Android 4.4 Kitkat feels pretty fast and stable and, despite the youth of this processor, I haven't run into any problems while using it.

However nobody should be surprised here, since Minix was the one that honestly repudiated releasing for its devices the embarrassingly fake "Android Kitkat Betas" that plagged chinese Rockchip sticks at the beginning of this year.


One thing that I found immature though was the H/W video decoding, since using MX Player I was absolutely unable to avoid seeing the subtitles in MKV movies, no matter how many forced/unforced subtitles checkmarks I removed. This does not happen when using a Rockchip TV box, so a newer Amlogic firmware should have to fix this.


The M1 airmouse is small, looks slick, and has an accurate response, I really like it.

However, when I removed the USB receiver from the X8-H, I found myself with the plastic cap in one hand and the electronic PCB still connected to the USB port... so better handle it with care.




The serial console


That said and, after fellow developer Omegamoon dared open it first and find the UART pins, I went about to do the same since, as you may already know, the UART (or serial console) is necessary for first tinkering with ROMs/Linux, due to being the place where all debug output is dumped to, way before you get anything on screen. And, of course, no kernel compilation works the first time, at least not for me :P

Apart from that, and until flash reading/writing through USB from a PC Linux is provided with tools by Amlogic, the best way to flash and reflash this device is to use the "u-boot" bootloader, through the serial console. U-boot is what runs after power-on but just before any kernel, Android or Linux, is loaded. In fact, u-boot is in charge of loading the kernel.

Hitting Enter on the serial console while the X8-H boots is enough to get a u-boot prompt instead of booting to Android.



PCB inside Neo X8-H


Just as its smaller brother, the Neo X7, the internals of this device boasts quality. And you may say: Why?

Well, let me recount a few items that you will NOT find on other cheap TV boxes that usually suffer from reboots in the middle of your favorite movie:

- The processor has a large heatsink of quality metal on it (instead of crappy tapes)
- The memory has that large heatsink also over it. What? The memory also has a heatsink!
- The large heatsink has another heatsink on top of it! What? Yes, attached to the top of the box!


Minix Neo X8-H PCB

Not enough?
- There is an embedded battery to keep the RTC time while powered off
- Count the number of capacitors, coarsely that's a major plus for stability.
- They even soldered pins for the UART serial console!! (bottom left)

Yes, the UART pins are right there for us tinkerers:

UART RX/TX pins plus power pins


So, with just a 3 pin [low-profile] connector broken in two, I was able to get them out:

Black for Ground, Red for X8H's TX, Blue for RX.

With a rather ugly cutout at the back of the box (upssss):




And, finally, connecting it to my PC through a Serial2USB board, the full boot log was available, including access to the u-boot console (boot log here).




Quick tests


The u-boot version is reported as this bizarrerie: U-boot-00001-g473771c-dirty
As far as I know it should be the year followed by the version within that year, so I guess somebody must have changed it, obscuring the source code it's based on.

u-boot log on Minix Neo X8-H

Once booted, Android reports kernel version 3.10.33, which is the latest one openly provided by Amlogic, and which, after some comparisons I believe is based on Linaro Stable Kernel tree for Android, which currently has the same version.

Unfortunately, I have found a few of the same device-specific kernel source modifications by Amlogic people, so typical of Rockchip kernels. That is ugly, prevents mainlining (using the official kernel and easily updating it), and is really badly looked upon.


On the bright side, this kernel is very modern and, magnificently, it is device-tree based.

What's that you say? Well, a device-tree file would allow you to use the very same kernel (no recompiling ever!) among all devices using, at least, the same Amlogic processor.
You just need to flash a different device-tree file! It's like an index telling the kernel what does the box in question have inside and where to find it.

Previously, on the dark ages that Rockchip sent us to, every vendor wildly modified the kernel source with its own, often buggy, little specifics, which caused an explosion of RK kernels everywhere, a chaos where you used a different kernel for each device and went crazy to keep them all up to date.



I have done some quick tests like extracting the device-tree file. Here you can see the device-tree of Minix Neo X8-H, which you can compare to the one posted by CNX-Software for the Tronsmart Vega S89 device-tree, it's interesting!


Another thing I've done is test the capabilities of u-boot, like being a client for a TFTP server on my PC, where I plan to place my kernel compilations, to download and flash to the Neo X8-H, like in this example:

u-boot getting new file for the X8-H from a TFTP server

I have been told that Minix may soon be releasing the source code its TV box is using, which is again one of the nice things we can expect from Minix, who has already released it before.

Once that is done I'll go on to recompiling and trying to get Linux into this beast! Stay tuned for the instructions!




Friday, January 3, 2014

Hacking into the Minix Neo X7


       Recently, just in time for the holiday season, I have been testing and developing with a Minix Neo X7 and its very handy A2 airmouse.
Minix NEO X7 TV box with RK3188 Quad-Core CPU and 2GB RAM


    This TV box is remarkable to me on the outside for the quality of its finishes, that do make it sturdy and good looking. On the inside there are some probably good choices on the design, namely:

- A quality Audio CODEC (RT5616 that afaik is a Realtek ALC5616) instead of the usual cheap RK1000 shipped on most TV boxes.

- A well suited on-board Ethernet MAC (Realtek RTL8152) as opposed to using the internal MAC of the RK3188, as is done by most other TV boxes, which are riddled with bugs (to list a just few of RK MAC's problems: quirks when connecting Ethernet directly to a PC, stops working on TV-box Linux restart, CPU shut-down upon serving heavy files over SAMBA, ...).

- An on-board battery for the RTC chip (real-time clock HYM8563 to keep the time even without network), which may be interesting for embedded projects with a Linux OS.


    Also, as a less precisely statement, though based on my electronics background, just by looking at the X7 PCB one can find many more capacitors than on most other devices. This is maybe one of the most important good points of this box, since having those capacitors means much more stability, so that it is far less likely for it to hang/reboot while doing heavy duty stuff.

Next time your Android stick/tablet/phone reboots, hangs, or throws you out of an app, think that more than half the times this is due to trimmed down capacitor count, while the rest of the times it's just buggy SW.




Apart from that, Minix has already released twice their kernel sources:

- In September 2013, to well known developer phjanderson (here)

- And then again in December 2013, for the MINIX Developer Challenge, which by the way may be of interest to Android hackers and artists, since they offer well rewarded prizes for the best UI (up until Feb. 9, 2014).





    The practice of respecting the GPL license is something we, as an open source community, value most and certainly distinguishes one vendor from another.

    Think that, even for Android, having kernel sources means an "explosion" of ROMs, developers, and, generally, features such as safer overclocking, updates, fixes (i.e. Vsync screen fix), etc, without the irksome binary patching.

     However, if I could make a wish, that would be the schematics of this TV box, so we can know which CPU pins are connected to which pins of the many ICs on the board, so as to give an even better support for all its components.
     Please note that plain schematics are like the blueprint on hotel room doors, they don't disclose how the hotel was built, just how the rooms are set up.
   

Hacking the machine


1) First things first, you may want to backup your recovery partition (no need to backup the others, unless you have your data on it, since MINIX offers the latest stock ROMs on their site). For that we do the following:
- Coonect the X7 to the power outlet (but don't press the power button yet)
- Connect the X7 to the PC through its USB OTG MicroUSB connector
- Carefully insert a clip into the "Recovery" pin hole (near the HDMI connector) until a pushbutton clicks, and keep it pushed.
- Press the X7's Power button

Then, upon doing a "lsusb" on the PC you'll see the X7 in bootloader mode, listed as a nameless USB device.

You can check the partition table (as a list formatted like: partition_size@partition_offset(partition_name)) by issuing this command:

sudo ./rkflashtool r 0 1 | head -n 11

The partition parameters is what follows after mtdparts. On my X7's revision this looks like:

mtdparts=rk29xxnand:0x00002000@0x00002000(misc),0x00008000@0x00004000(kernel),0x00008000@0x00012000(boot),0x00010000@0x00020000(recovery),0x00020000@0x00030000(backup),0x00040000@0x00050000(cache),0x00800000@0x00090000(userdata),0x00002000@0x00890000(kpanic),0x00130000@0x00892000(system),-@0x009c5000(user)

So backing up recovery partition becomes (swapping size <=> offset):

sudo ./rkflashtool r 0x20000 0x10000 > NEO_X7_recovery.img


2) Second step would be to root Android, so you can reboot to recovery (i.e. Linux) anytime. Rooting the X7 is done as described in this post.



3) Now you want to compile your kernel with the options (.config) you prefer, just remember to use the latest kernel source, at the last line of the MINIX Developer Challenge announcement.

As a base for any hacking, the stock Neo X7 config is: /arch/arm/configs/box_3188_r2_defconfig

To compile your own kernel you can follow this post: Generic Linux Kernel building for RK devices.



4) Last step is flashing your shiny new kernel to the recovery partition of the X7. This is done with the following two commands:


sudo ./rkflashtool w 0x20000 0x10000 < recovery.img
sudo ./rkflashtool b


5) In order to boot to Linux, you just need to insert an SD card with your Linux RFS (you can create your own RFS as described here) and then, you can either use this excellent "Autorun Linux" app, or install "Android Terminal Emulator" get inside it, type "su" and then "reboot recovery" every time.

That's it, happy hacking!




Regarding Android 4.4 KitKat


Yesterday Minix issued a brief message regarding the recently announced, supposed, Android 4.4 support on some RK3188 devices from other vendors [that do not release their kernel sources].

Since I have been myself, along with other well known developers in the RK Linux community, doing work on upgrading the RK kernel to newer versions (3.0.36+ is tremendously outdated), I can confirm Minix' point of view, that the supposed 4.4 support by those other vendors is indeed FAKE.

The leaked 4.4-supporting kernel is the same old 3.0.36+ kernel with just some drivers in different structures/places and the bare minimum to trick Android 4.4 into believing it is running a newer kernel.
Naturally this is a recipe for bad outcomes, instability and developer support dispersion.

It kinda feels like those MicroSD cards that are labelled 8 GB but are internally just 4 GB...

So, I really look forward to the official release of a real Android 4.4 and its kernel sources. Until then I think we are better off with a nice theme in our current and more stable Android Jelly Bean.



Sunday, December 1, 2013

OpenSUSE for Rockchip ARM devices



      Going on with the earlier post on creating your own MicroSD with the Linux RFS of your preference, here is the howto for installing OpenSUSE using the usual kernel and modules+firmware.
      Just in case, whatever your stick/box, you can use these very generic 3.0.36+ kernels for RK3066 or RK3188 (only missing internal Wifi).

       In my case I used the excellent Radxa Rock board, equipped with a quad-core RK3188, for the tests and found no problem booting it:

OpenSUSE on Rockchip RK3188 (Radxa Rock board)



     Of course, this has been made possible by OpenSUSE team's commitment to the ARM architecture, since they are one of the few that have put all the necessary efforts to have their distribution compiled for ARM CPUs.

     OpenSUSE's official wiki has one page dedicated to installing on a RK3066 device (MK808 [1]), though it is outdated since, among other things, it uses the old 3.0.8 kernel. However the guide was useful and I wanted to update it here.


Now the procedure, with the same captions as in the original post (for Linaro Ubuntu), for comparison:



Getting an up-to-date official distribution, for ARM
 
     You can check the latest ARM distribution at OpenSUSE by heading to this url:
http://download.opensuse.org/ports/armv7hl/distribution/openSUSE-current/images/

and downloading the "rootfs" suffixed .tbz file, which at the time of writing (Dec 2013) is this one:
http://download.opensuse.org/ports/armv7hl/distribution/openSUSE-current/images/openSUSE-12.3-ARM-XFCE-rootfs.armv7l-1.12.1-Build49.1.tbz




Getting the RFS into the MicroSD card (or flash)

Exactly the very same procedure as in original post, except that steps 4 and 6 are NOT to be done, and step 3 becomes:

3) Extract the downloaded ARM RFS in it:
tar xjfv /home/username/Downloads/openSUSE-12.3-ARM-XFCE-rootfs.armv7l-1.12.1-Build49.1.tbz 


And this last step is necessary (or else the root file system is mounted as read-only, preventing correct boot), open the /mnt/whatever_folder/etc/fstab in your linuxroot and add the following first line:
/dev/root / ext4 defaults,noatime 0 0



Getting a desktop environment

    You already have it. However I've found a problem: upon opening the "Install/Remove Software" you will likely stumble upon this fatal error:

"UI Syntax error. Couldn't load plug-in gtk-pkg. Check the logfile"
Solution is simple [2], open a terminal and type:
su -c "zypper in libyui-gtk-pkg4"
You'll be asked the root's password and the missing package installed, so now you can open the Install Software app.


Enjoy!




References:
[1] http://en.opensuse.org/HCL:MK808
[2] http://forums.opensuse.org/english/get-technical-help-here/applications/487505-opensuse-samsung-arm-chromebook-almost-works.html#post2561668

Wednesday, November 27, 2013

Your own official Linux distro in a SD card (for ARM)

Currently for us, owners of ARM devices of the Rockchip family (though this post applies to any ARM CPU), when wanting to boot Linux we have to do two things:

1) Flash a Linux kernel, usually to recovery, in order to be able to boot to Linux
2) Copy a Linux Root File System (RFS) into a MicroSD card for the OS to boot from

This post is about Step 2.



Why can't we use official distributions?


Basically because most distros are [easily] available for x86 CPUs (x86, amd64, ...), not ARM CPUs (armel, armhf). And then, when you find it compiled for ARM, there are some little extras to be done, like copying your kernel's modules into it, and throwing it all into a MicroSD card.

Hence, we are currently forced to choose among static RFS that fellow developers have made available and can update only on their free time, which isn't much. These are:

- Picuntu, from Alok Sinha
- Ubuntu 12.10 RFS, from linuxium
- Home.io, from JustinTime4Tea


These RFS are all just the Ubuntu distribution, sometimes with a different name (Xubuntu in the case of Picuntu, Ubuntu in the case of Home.io), though they usually include extras like:

- The modules (drivers) for many devices (needed for many, but not all, USB gadgets)
- The Flash video ARM support solution I posted here

However let's face it, you are forced to use Ubuntu, and then you depend on the maintainer's free time to grab the latest version of Ubuntu, add the above extras and publish it so you can use it.



Your own Linux distribution's RFS


Here I am going to show how to create your own RFS by grabbing whatever version of Ubuntu (and with few changes any other distro, like Arch Linux, OpenSUSE, CentOS, just click on the link to be taken to the specific distro's instructions), and create a RFS to use on a MicroSD card to boot Linux your ARM stick/box. Or to flash the same RFS into your device's Nand Flash chip and avoid the SD card altogether.




Getting an up-to-date official distribution, for ARM

The first thing is getting the Linux distribution (all the programs that make up your booted up Linux: from the "ls" commandline, to the GNOME desktop and the LibreOffice suite) already compiled for ARM devices, instead of the usual x86 for PCs.

This is not difficult since, actually, Linaro is doing just that!
If you want a desktop environment (XFCE, Gnome, KDE, etc.) you have two options:

Option 1) Easy way: (Older version but everything done) Head over to:
http://www.linaro.org/downloads/
Look for "Ubuntu Desktop" in the "Developers and Community Builds" section and click on any of the boards that support it on the right (Origen, Panda, etc. doesn't matter). At the time of writing (Nov 2013) this will redirect you to this page:
http://releases.linaro.org/12.11/ubuntu/precise-images/ubuntu-desktop
So the latest version with the desktop already installed is Ubuntu 12.11, you'll have to install this one in your SD and upgrade to the latest version after booting. Download the biggest .tar.gz file, in my case 473 MB of:
http://releases.linaro.org/12.11/ubuntu/precise-images/ubuntu-desktop/linaro-precise-ubuntu-desktop-20121124-560.tar.gz
Option 2) Manual way (Newer version, but starts with a commandline Linux): You can get the cutting edge latest version of Ubuntu, it will be small and fast but you'll have to type a few more commands, after booting it, in order to install a desktop. Go here:
http://releases.linaro.org/
And click in the folder link of the latest version number ("13.11" for me). A new page appears where you have to select "ubuntu", in the next page select the link that ends in "-images" (prefix depends on Ubuntu's version name, in this case it's "raring-images"), and finally select the version link that you want (nano / developer / server) to get its .tar.gz file.

All three are the same Linux but each one with different sets of packages already installed. They are:

- nano (around 50 MB) gives you a command-line with very very few things (not even a vi/vim/nano editor!), with this version you'll need a network connection (care for /etc/network/interfaces and such) soon enough, to start cranking "apt-get install" for all the little apps.

- developer (around 140 MB) a usable command-line with most things you'll need to comfortably start working and changing whatever configuration. However, IMHO, it includes many packages for software development, while lacking some others more typical of a generic user system.

- server (around 130 MB) a usable command-line with all you need for a quick setup of a desktop environment resembling a PC with Ubuntu. Even if you are a developer you can later install all the packages you need. This is the one I recommend.

N.B.: Compare the *.packages files to get a glimpse for yourself of what's inside each version.

So, following the recommendation we would download this file for the latest Ubuntu 13.11 (Nov 2013, or else go into the page above and get yourself the latest and very best.




Getting the RFS into the MicroSD card (or flash)

First of all introduce the MicroSD card into your PC to partition it in the way that the kernel expects it (please don't use partition utilities, they won't do it!).

Let's say that the MicroSD appears at the folder "/mnt/whatever_folder", then type the following command to know the device name:
df -h | grep whatever_folder
You should see a line somewhat like this:
/dev/sdg1   7,3G   612M  6,4G   9% /mnt/whatever_folder
Now that we know the MicroSD is in "/dev/sdg" (remove the number!!) unmount the MicroSD:
sudo umount /mnt/whatever_folder
And the most important step to avoid later problems: partition & format in this one step:
mkfs.ext4 -F -L linuxroot /dev/sdg
Which labels it "linuxroot" and at "/dev/mmcblk0", where the kernel expects it (CMDLINE), instead of where a partition tool would place it (at "/dev/mmcblk0p1").

SIDE-NOTE ONLY FOR USING INTERNAL FLASH INSTEAD OF A MICROSD:
If you want your RFS to be in your device's Flash chip, and 1) have flashed the right parms, 2) have a kernel with the right CMDLINE for those parms, 3) the kernel has an initramfs.cpio with rknand...ko, then the you have to format it with this command: "mkfs.ext4 /dev/mtdblock0" (mtdblock number is that of your [big] partition).

It may take a couple minutes and when finished you should mount it and start following these steps:

1) Become root (necessary to keep file permissions in the RFS!)
sudo su -
2) Go to the folder where the SD card (or flash) is mounted:
cd /mnt/whatever_folder
3) Extract the downloaded ARM RFS in it:
tar xvfz /home/username/Downloads/linaro-raring-server-20131124-562.tar.gz
4) Since the Linaro .tar.gz contains the RFS inside a folder named "binary", we have to move its contents to the real root of the SD card and then remove the, now empty, "binary" folder:
mv binary/* .
rmdir binary
5) If you are doing this process on a PC, you have to uncompress the modules+firmware file (that came with the kernel you flashed into your stick) into the "/mnt/whatever_folder/lib/" folder.


Or, if you are on an ARM stick, you can just copy your own modules (drivers) and firmware to the new RFS so you have everything from the beginning:
mkdir ./lib/modules
cp -R /lib/modules/* ./lib/modules
cp -R /lib/firmware/* ./lib/firmware
And it won't hurt to copy the library for playing Adobe Flash videos (think YouTube) in ARM (and all the Flash ads and websites, of course):
cp /usr/lib/libpepflashplayer.so ./usr/lib/
If you didn't have it, follow this quick post.


6) You may also want to "touch" certain files to make sure networking works out of the box. For example, if you have an Ethernet connection in your ARM device, then modify the /mnt/whatever_folder/etc/network/interfaces file to add these lines:
auto eth0
iface eth0 inet dhcp
7) It's always a good idea to end operations with this command (to finish all write operations):
sync

You can unmount that MicroSD, because it's ready to boot. If you installed the desktop version the easy way, that's it.
However, if you installed a trimmed commandline version the manual way, you should continue reading the following section.



Getting a desktop environment


Once you boot your device with the created MicroSD card, your first task is to check the network connection. For example type:
apt-get update
If the software repositories start updating, you're good to go. If there is no connectivity: Internet is full of Q&A about networking in Linux :)

Now, with Internet on, in order to install a desktop environment, in my case XFCE (the one used in Xubuntu), I would just type this:
apt-get install xubuntu-desktop
And when asked, say Yes to downloading >400 MB of packages that will use up >1 GB of space when installed, but will land me on the very latest desktop environment for an ARM PC, whatever your CPU (Rockchip, Allwinner, Samsung, ...).




Sorry if it took too long to explain something that is actually simple, but everybody is welcome!
The more users discover that a tiny and cheap stick/TV-box is actually a full-fledged low to mid-end PC (without the noise, size, and the Watts!), the better for all!


Tuesday, November 19, 2013

Linux on UG008 TV Box (RK3066)

I was looking for the cheapest Rockchip device with external Wifi antenna and/or Ethernet connection, just to find both capabilities in the UG008 TV Box, a RK3066 (dual core) device at just 55 USD (shipping included).

After purchasing it and a quick delivery of 2 weeks to Europe, I set out to test it, since I worried it may be lagging behind the newer 4 core experience with the RK3188 CPUs.

The TV box surprised me being very small, at 7.5cm x 7.5cm and looking really slick and solid. I was glad to see Android feels zippy.


Small and cheap UG008 TV Box (wifi antenna is also just 7 cm)


So, hacking started:

1) Backup internal flash: The recovery button is at a tiny hole in one side, near the front. The only quirk here is that you insert a clip there and stay pressing the recovery button, plug the MicroUSB from the PC to the TV box and then: you also have to press a couple of seconds the power button, before you can release the recovery.

A red light appears around the MicroUSB slot, and on your [Linux] PC you can type "lsusb" and find a nameless device (for my PC's buses/devices, it is: "Bus 002 Device 004: ID 2207:300a  "), the 2207:300a ID is what matters, that is our UG008.

First's first, the "partition table", extracted with:
sudo ./rkflashtool r 0 1 | head -n 11
looks like:
CMDLINE: console=ttyFIQ0 androidboot.console=ttyFIQ0 init=/init initrd=0x62000000,0x00800000 mtdparts=rk29xxnand:0x00002000@0x00002000(misc),0x00006000@0x00004000(kernel),0x00006000@0x0000A000(boot),0x00008000@0x00010000(recovery),0x00120000@0x00018000(backup),0x00040000@0x00138000(cache),0x00300000@0x00178000(userdata),0x00002000@0x00478000(kpanic),0x00120000@0x0047A000(system),-@0x0059A000(user)
We go one by one backing them up into our PC (you can safely leave out some, like user, cache, kpanic), just remember for the rkflashtool command you have to swap the hex numbers above.

For example to backup recovery (to flash our Linux kernel on it), which is listed above as "0x00008000@0x00010000(recovery)", you'd do:

sudo ./rkflashtool r 0x10000 0x8000 > UG008_recovery.img

2) Rooting UG008: Once we are done with that, we should root the device (even if just for the sake of being able to dual boot Android or Linux), the rooting procedure is fairly standard and described here. If you only want Linux on the box, you don't need to root it, keep reading,


3) Flash Linux kernel: You have two options depending on what you want:

a) Dual Boot Android/Linux: This is the usual way, you want to flash the recovery partition. I've found this box to be a close relative of the Measy U2C, for which I published several recovery kernels that are compatible with UG008, you'd be missing Wifi (RK901) and Ethernet. Flashing being just:

sudo ./rkflashtool w 0x10000 0x8000 < recovery.img
sudo ./rkflashtool b 

      If you're into working your way around, it should be easy to add the connectivity. Wifi with RK901 has been supported in Linux sticks for some time already (though I never bothered to check myself, being used to wired LAN), and Ethernet should require only to enable this .config options before building your kernel: CONFIG_NET_ETHERNET and CONFIG_RK29_VMAC.

Generic Linux Kernel building for RK devices is described here. Just remember this is a RK3066 device, so don't compile it for RK3188! The audio codec is the well known RK1000.

Now, in order to boot to Linux from Android you can use this excellent "Autorun Linux" app, or install "Android Terminal Emulator" get inside type "su" and then "reboot recovery" every time.


b) Only Linux (remove Android): Very legitimate. Using the same recovery kernel as above, the flashing procedure is all explained here (but follow the instructions for flashing "boot" partition, instead of "kernel" partition, since you are using a recovery kernel). The boot writing (after you back up that partition!!) can be done with this command:

sudo ./rkflashtool w 0xA000 0x6000 < recovery.img
sudo ./rkflashtool b 

Note: I haven't tested the "Only Linux" approach on my UG008, though it should work, or else you can always flash again your backed up stock partition and be back where you started.



4) Booting Linux: Insert your usual MicroSD card with Linux root file system (rfs) and you're good to go.


Any questions, and polite criticism is welcome in the comments below. I hope this post helps!