Compiling Swift from Source on the Raspberry PI

In late 2015, Swift was fully open sourced by Apple and hosted by GitHub. (you can find more info about both of these items on the Swift.org website and Apple’s Github.)

What this did was open up a lot of interesting opportunities and potential for a lot of people / projects / systems. I’m sure the first thing on a lot of people’s minds was… “will it run on X?”. Well the answer is… sure why not… Apple already provide ready-made packages for a few Linux variants and there are also ready-made packages out there for other platforms too, including android :o. There’s no fun in using a ready-made packages though right? if you’re insane enough, why not try to compile Swift on the Raspberry PI? Well thats what I did and here’s what I found out…

If you want to get a decent installation of Swift up and running, including the compiler, Foundation, stblib then the whole thing consists of 9 different git repo’s.

Initially I tried a Raspberry PI Zero ($5 Tiny Computer) running the latest Raspbian. Unfortunately, after trying a few times (with each cycle taking around 12 hours), I didn’t get very far at all. First I had to make some modifications to some of the repo compile time checks, then manually install newer versions of cmake / clang etc, try different compilation configurations etc… after which the PI Zero sadly didn’t have enough memory to get to the very end. So for the remainder of this post i’ll be using a Raspberry PI2 running the latest Ubuntu Mate image.

From here-on in, i’m gonna summarise a little, as getting this far took me a few days of hacking combined with trial and error, of which i’ll spare you the details.

 

Ubuntu Mate¬†This is the OS you’re gonna need to load onto your microSD before continuing. You could try with Raspbian, but i’ve had mixed results. NOTE: this only works on armv7 based PI… so as of writing thats the Raspberry PI2.

A full clone of all repo’s you’ll need for Swift, as you’ll see below, a couple of the repo’s are not the official Apple ones, this is because they’ve had patches applied to them to aid in compilation on the Raspberry PI

UPDATE: 2017, You can now check-out all the repo’s directly from Apple and it will successfully compile as PI support has been merged in.

sudo apt-get install git -y
git clone https://github.com/apple/swift.git swift
git clone https://github.com/apple/swift-llvm.git llvm
git clone https://github.com/apple/swift-clang.git clang
git clone https://github.com/apple/swift-lldb.git lldb
git clone https://github.com/apple/swift-cmark.git cmark
git clone https://github.com/apple/swift-llbuild.git llbuild
git clone https://github.com/apple/swift-package-manager.git swiftpm
git clone https://github.com/apple/swift-corelibs-xctest.git
git clone https://github.com/apple/swift-corelibs-foundation.git

In case you still want to use the old repo’s, they’re here for reference.

sudo apt-get install git -y
git clone https://github.com/hpux735/swift.git swift
git clone https://github.com/apple/swift-llvm.git llvm
git clone https://github.com/apple/swift-clang.git clang
git clone https://github.com/hpux735/swift-lldb.git lldb
git clone https://github.com/apple/swift-cmark.git cmark
git clone https://github.com/apple/swift-llbuild.git llbuild
git clone https://github.com/apple/swift-package-manager.git swiftpm
git clone https://github.com/apple/swift-corelibs-xctest.git
git clone https://github.com/hpux735/swift-corelibs-foundation.git

Make sure to clone all of the above into the same parent folder, so you should end up with 9 different folders.

There are a few dependencies you will need to get the compilation going, along with a few others added in for your benefit, as you’ll need them immediately after.

sudo apt-get install build-essential cmake ninja-build clang python uuid-dev libicu48 libicu-dev icu-devtools libbsd-dev libedit-dev libxml2-dev libsqlite3-dev swig libpython-dev libncurses5-dev pkg-config

Now comes the more nitty gritty part. You’re gonna need three resources for this compile. CPU, RAM and Disk Space. You can’t do a lot about CPU, however you can improve the RAM and Disk Space situation quite easily.

Assuming you haven’t already expanded your root filesystem to the extents of your microSD card, we’ll do this now using the easiest approach possible. There’s a neat little utility called raspi-config that is an aid to some of the common tasks you’ll need to do on your pi.

sudo apt-get install rasps-config -y

Now run it and choose “Expand Filesystem”, after you’ll see an option to change the Memory (RAM) Split between GPU and System, you’ll want to allocate as much as possible to the System for now, it’ll make the GUI a little sluggish but don’t worry about that for now… After you’ve changed the value reboot and thats it!

After all this, we still have a lack of RAM… likely around 1012MB (1024-12vmem) available. The compile of Swift will take a lot of memory to get things going, so we’re gonna enable a SWAP file. This essentially will allocate a portion of your microSD card for on-demand “RAM” that can be used when your actual RAM has ran out. Its very similar to how paging works on other platforms, but at the same time, completely different. The caveat is… SWAP on a microSD is very very slow, however you’ll have to make do.

sudo apt-get install dphys-swapfile

This will enable a file to be used for SWAP, usually on Linux you slice off a specific partition for this purpose, but as its only going to be temporary whilst we compile Swift, we’ll use our approach.

Now that all the above is complete, we’re ready to get on with compiling ūüôā We’re going to build a “release” configuration of Swift with no-assertions enabled, what this will do is greatly speed up the compile time (by around a factor of 2) whilst reducing the memory usage required. The trade-off is that you won’t have debug symbols or a great ability to poke around. For our needs, it’ll suffice.

From the parent folder you cloned the 9 repositories into, perform the following steps:

cd swift
utils/build-script -R --no-assertions

Now go and make a tea, watch a box-set of game of thrones and come back around 9 hours later… You’ll either find yourself with an Error, or a lovely compiled version of Swift on your Raspberry PI! If the latter is the case, lets take it for a spin!

~# nano helloworld.swift
  let message = "Hello World!"
  print(message)

hit (Ctrl+X)
hit (Y)

~# swiftc helloworld.swift

~# ./helloworld

Hello World!

Viola!

P.S I’m currently refining this post to add a script that performs all of the above.

Questions or comments, hit me up on Twitter @ArmstrongAtWork

 

SuperRecord an ActiveRecord implementation in Swift

All the media buzz around swift seems to be having some tangible effect. I won’t go into the opinionated slaw of Swift VS Objective-C VS Swift VS XYZ here :). Many of you know I freelance in the iOS space around London and beyond for SuperArmstrong and one of my recent clients asked me to work on a greenfield project to replace a very old wrapped web app they had. When we got down to the nitty gritties, one of their few requirements was that it should be written in Swift. My jaw did drop a little… considering this was about 20 days after Swift had first been publicly announced. After some conversation, we came to the mutual conclusion that they were brave.

Like many of the apps I work on, this project relied heavily on CoreData, as a convenience I decided i’d try out a library that I have a lot of respect for (even though its had its challenging moments) “MagicalRecord”, having only done organic¬†CoreData for the past few years. As you probably know, Objective-C and Swift can sit alongside in a project and work in unison… however, not all the interoperability¬†is 100% sound. I soon started to endure strange bugs and thanks to the incompleteness of Swift lldb support at the time (especially when mixing with Objective-C) these bugs soon became quite difficult to track down.

The most annoying was this:

let entityDescription = NSEntityDescription.entityForName("Pokemon", inManagedObjectContext: context)

Although harmless… this would actually fail when trying to insert an entity of that description into your managed object context. The culprit here? the NSString to String bridging (or vice versa).

To get around this bug (which I won’t go into too much detail) I began using:

let entityName = "Pokemon" as NSString
let entityDescription = NSEntityDescription.entityForName(entityName, inManagedObjectContext: context)

Performing a forced downcast to NSString. So great, now it works… However, many of the MagicalRecord finders and helpers I was using, would not have these forced downcasts as they came from the Objective-C world. I first forked MagicalRecord, after delving into the source-code, making many changes for better Swift support, fixing some threading irregularities I then realised I was putting a lot of work into making something work that had more functionality than I required at the time. So I embarked on making a Swift ActiveRecord style “companion” to CoreData.

The reason I say “companion” and not “wrapper” is that I didn’t want to abstract away CoreData too much or remove the power from the hands of the developer. I wanted to make the developer’s life easier, whilst maintaining their flexibility.

I’d much rather do something like this:

let pokemon = Pokemon.createNewEntity() as Pokemon

and avoid the above lines + more.

Thereby SuperRecord was born. The original goals for SuperRecord were simple.

  1. Give me the ActiveRecord style finders I crave, optimise them, keep my application level code tidy, but still performant.
  2. Rather than me typing out the same code in each project to handle batch updates on my UITableView and UICollectionView classes, have a special “one size fits all” class that will act as your NSFetchedResultsControllerDelegate, providing “safe” batch updated to your reusable views.
  3. Additionally, I didn’t wanna have to spend too much time creating NSFetchedResultsControllers all the time either, so I added some helpers for this too.
  4. Be written in Swift, keeping the public API’s simple and stable, but changing the implementation as Swift changed and moved with updates.

In late October 2014 whilst still very much a work in progress, I released the first public version and it was featured in the great iOS Dev Weekly newsletter. I decided to release it early as I saw a lot of discussion around CoreData and Swift and thought there’d be a lot of people interested in this OSS effort, so we could build up a good toolset as a community. I’m still the only maintainer… (not by choice) but its early days :).

Below is some extract from the README.md but I suggest you head over to the project on github for more information and feel free to checkout the demo project also on github which shows how to use the safe batched updates along with some other common tasks.

 

Core Files

  • NSManagedObjectExtension.swift¬†This extension is responsible for most of the “finder” functionality and has operations such as¬†deleteAll(),¬†findOrCreateWithAttribute()createEntity()¬†and allows you to specify your own¬†NSManagedObjectContext¬†or use the default one (running on the main thread).
  • NSFetchedResultsControllerExtension.swift¬†In constant development, this Extension allows the easy creation of¬†FetchedResultsControllers¬†for use with¬†UICollectionView¬†and¬†UITableView¬†that utilise the¬†SuperFetchedResultsControllerDelegate¬†for safe batch updates.
  • SuperFetchedResultsControllerDelegate.swift¬†heavily inspired by past-projects i’ve worked on along with other popular open source projects. This handles¬†safe batch updatesto¬†UICollectionView¬†and¬†UITableView¬†across iOS 7 and iOS 8. It can be used on its own with your¬†NSFetchedResultsController¬†or alternatively, its automatically used by the¬†NSFetchedResultsControllerExtension¬†methods included in¬†SuperRecord.
  • SuperCoreDataStack.swift¬†a boilerplate experimental main thread CoreData stack. Can be used either as a sqlite store or in memory store. Simply by calling¬†SuperCoreDataStack.defaultStack()¬†for SQLite or¬†SuperCoreDataStack.inMemoryStack()¬†for an in memory store. Of course you have access to your context¬†.context¬†/¬†.saveContext()

Usage

Create a new Entity

Assuming you have an NSManagedObject of type “Pokemon” you could do the following

let pokemon = Pokemon.createNewEntity() as Pokemon

Please add @objc(className) above the class name of all your NSManagedObject subclasses (as shown in the demo project) for now. Better support will be coming in the future.

Creating an NSFetchedResultsController

This feature is currently in progress with basic support so far, in future versions, sorting and sectionNameKeyPath’s will be supported. Until then you can create your own NSFetchedResultsController, however, if you have no need for the above missing functionality then simply use

lazy var fetchedResultsController: NSFetchedResultsController = self.superFetchedResultsController()

func superFetchedResultsController() -> NSFetchedResultsController {
return NSFetchedResultsController.superFetchedResultsController("Pokemon", tableView: tableView)
}

With Pokemon being the entity name of your NSManagedObject.

Delete Entities

I’m planning on adding much more powerful functionality around Delete soon, such as deleteAllWithPredicate() or deleteEntity(), right now all that is available is

Pokemon.deleteAll()

Method Listing

This isn’t an exhaustive list of all methods and classes, however it includes some of the most useful ones.

  • NSManagedObjectExtension
  • findAllWithPredicate(predicate: NSPredicate!, context: NSManagedObjectContext) -> NSArray
  • findAllWithPredicate(predicate: NSPredicate!) -> NSArray
  • deleteAll(context: NSManagedObjectContext) -> Void
  • deleteAll() -> Void
  • findAll(context: NSManagedObjectContext) -> NSArray
  • findAll() -> NSArray
  • findFirstOrCreateWithPredicate(predicate: NSPredicate!) -> NSManagedObject
  • findFirstOrCreateWithPredicate(predicate: NSPredicate!, context: NSManagedObjectContext) -> NSManagedObject
  • createNewEntity() -> NSManagedObject
  • findFirstOrCreateWithAttribute(attribute: NSString!, value: NSString!, context: NSManagedObjectContext) -> NSManagedObject
  • findFirstOrCreateWithAttribute(attribute: NSString!, value: NSString!) -> NSManagedObject
  • NSFetchedResultsControllerExtension
  • superFetchedResultsController(entityName: NSString!, collectionView: UICollectionView) -> NSFetchedResultsController
  • superFetchedResultsController(entityName: NSString!, tableView: UITableView) -> NSFetchedResultsController

NSFetchedResultsControllers created using this method will automatically handle safe batch updates.

Developer Notes

This whole project is a work in progress, a learning exercise and has been released “early” so that it can be built and collaborated on with feedback from the community. I’m using it in a project I work on everyday, so hopefully it’ll improve and gain more functionality, thread-safety and error handling over time.

The next key things to be worked on are Optionality (as this has changed in every Swift BETA), the CoreDataStack, adding more finders with more functionality and improving the NSFetchedResultsControllerExtension.

 

 

Installing Debian or Ubuntu on a Cubox i4-Pro

IMG_0434In March I got my hands on a new Cubox i4-Pro, for those who don’t know what it is… its basically an armv7l based “raspberry-pi style” 2″ cube on crack! It packs a Quad Core 2GHz armv7l i.MX6 ¬†variant¬†architecture with 2GB of memory, on board bluetooth,wifi,esata,usb and more. Its pretty damn cool… Until recently I didn’t really have time to get it all up and running, but after unboxing it last week, I realised… damn this thing is tiny and must pack a punch… So I get it out the box and I wonder… so…. erm…. how do I get something running on this thing. After a quick lazy trawl of Google I noticed there isn’t a lot of info compared to other dev-boards and systems and a lot of info seems out of date. So here is my attempt to making a first post of a few, this one aimed at getting you up and running immediately on a CuBox i4-Pro with a “no-frills” approach… Next in line, i’ll maybe do some posts on how to¬†build your own image, kernel and more. These instructions are for Linux / Mac OS X only really. Those using Windows, google is your friend :).

  • Firstly download the latest debian jessi image from solid-run.com. (note: if you browse around there you’ll find both an ubuntu and geexbox image also).
IMG_0435 IMG_0436

Now you’ve got all that downloaded. The next step is to extract it using the ‘xz’ utility. Much like any other compression utility, this one is a little different as it uses lzma but supports the usual commands.

xz --decompress debian-jessi-4-july-2014.img.xz

You should now be left with a .img file thats ready to copy over to your Micro SD card ūüôā To do this we’ll be using the ‘dd’ utility that ships with most variants of Linux and Mac OS X. Windows users can find `windd` via the google link above ūüôā

But before hand… you’ll need to find out which device on your system is your Micro SD card and make sure NOT mounted!

Mac OS X :

~# diskutil list
/dev/disk0
#: ......
/dev/disk1
#: ......
/dev/disk2
#: TYPE NAME SIZE IDENTIFIER
0: FAT *31.97GB disk2

~# diskutil unmountDisk disk2

Linux :

sudo fdisk -l
sudo umount /dev/somedevice

Now you know which device you want… its time to copy over that .img! For this we’ll be using ‘dd’. Depending on which version of ‘dd’ you’ve got on Mac OS X or Linux, the command will vary very slightly, but remain very similar.

Mac OS X:
sudo dd if=/path/to/debian-jessi-4-july-2014.img of=/dev/disk2 bs=16384

Linux:
sudo dd if=/path/to/debian-jessi-4-july-2014.img of=/dev/sdb bs=16M

On Linux you’re likely to see progress, but on Mac OS X, nothing will “appear” to happen and the command prompt will hang, but trust me… thats a good thing, its copying… Once done. Your prompt will return to normal and after a few seconds, you can safely pull the Micro SD out of your machine.

Now place that Micro SD into your Cubox i4-Pro upside down. Plug in some power and Ethernet… and boot! ūüôā

 

Objective-C iOS Code Style Guidelines (Part #1)

I move around on different projects a lot and work with different developers every few months. One thing I’ve noticed is people picking up bad habits from blogs and Stack Overflow posts that get very popular despite their quality (this is not a blanket statement of course)… On the other hand however, I’m able to share experiences with other developers and pickup some good tips in the process. One bone picking point is always what convention to use, when, where and most importantly… Why!

Something i’ve been asked for a few times recently and something I continually tweak as my own preferences and style changes is my Objective-C iOS Code Style Guideline doc. I thought i’d share it here for everyone to use, love and consume. All I ask is that you don’t steal it as your own and that you perform any discussion on twitter using #mike.kz or message me @italoarmstrong ūüôā

These guidelines of course are my own personal preferences, which have developed over many years and will continue to change, even as writing this point, I changed my mind on a few points in the below guide. So take inspiration from it, rather than thinking “this is correct way of doing it” as its not. Thats usually the point in a “guideline”.

… 

 

How to setup Ruby on Rails on Ubuntu Linux

There are a lot of guides that do exactly the same as what i’m about to write about… the reason for this quick blog post is mostly a brain dump for myself.. but also because I always struggle to find up to date info for all of this in one place… and there are always gaps… So here goes! by the end of this short tutorial, you should have not only a working rails and ruby installation, running as a user, but also a ruby on rails hello world! whaaaaaat. … 

 

Reading Linux extfs (ext2/3/4) in Mac OS X

I have a collection of both Mac OS X and Linux servers dotted around the place… Sometimes I need to read ext4 formatted drives on my Mac’s… this always proves an annoying problem where I have to mount them inside a Virtual Machine and copy things across…. Pain and Slow!

I searched the interweb for a solution to this… and most ¬†guides are about 3 years out of date, so here’s one from 2013 that works on Lion / Mountain Lion and even Mavericks DP1/2.

Download and Install : OSXFUSE-2.5.4.dmg (this took off where MacFuse left off and even has compatibility for old MacFuse plugins).

Download and Install : fuse-ext2-0.0.7.dmg

Plug in your ext fs formatted drive… Voila! I wouldn’t rely on this in a production environment mind you… but its decent if you just need to grab some files from a USB drive quickly for example.

 

Ubuntu 13.04 on a Late 2012 Mac Mini

EDIT: Tonight i’ll be picking up one of my Mac Mini’s from the Datacenter to get Ubuntu 13.04 up and running! expect a full guide with drivers here shortly ūüôā

So Ubuntu 13.04 LTS recently was released, It comes with the new 3.8.0-19 upstream of the Linux Kernel so I thought I’d check it out!

Although our patched 12.04 and 12.10 Ubuntu’s use version 3.124c of the tg3 NeXtreme drivers from Broadcom which have Mac Mini support… The version in Ubuntu 13.04 (3.128c) seems to have had this removed!

A simple run of modinfo tg3 | grep 1686 reveals sadly that support for detection of the Mac Mini Ethernet hardware seems to have been removed during 3.124 and 3.128 of the Broadcom tg3 drivers.

I’m likely to install 13.04 on a Mac Mini sometime soon so will update this post with a proper howto and any good news I encounter but I don’t think its good news…

lsmod | grep Ethernet returns
01:00.0 Ethernet controller: Broadcom Corporation Device 1686 (rev 01)

whilst modinfo tg3 | grep 1686 on our modified 12.04/12.10 machines using the NeXtreme driver from this blog returns:

alias:          pci:v000014E4d00001686sv*sd*bc*sc*i*

however on 13.04 returns nothing.

 

Getting started with Mosh: the mobile shell

Mosh is something i’ve been meaning to give a go for a while now… but have never had the chance. Recently however, on a train from Rome to Florence, I got a little tired of reconnecting SSH sessions and re-attaching screen’s / tmux’s. Then I remembered… Mosh!

Some examples of why / when to use Mosh

Dubbed “the mobile shell” mosh is simple, it requires you have the mosh binaries installed on both client, and server, and that you have permission to SSH to said server (you don’t need a root account). Then mosh will create an initial SSH session, which it in-turn uses to create a UDP connectionless “session” between mosh running on the server and mosh running on the client.

Why?

Well… I can type mosh username@server.tld start a nice terminal session, close my laptop / phone, come back 4 hours later, open my laptop… and carry on. It’s also very useful for environments where your connectivity is poor or you have a dodgy ISP supplied Router that can’t sustain SSH connections very well.

$ mosh username@hostname.tld

Directly, stolen inspired from the original website. Mosh will log the user in via SSH, then start a connection on a UDP port between 60000 and 61000.

Beyond just having a really stable “connection” to a remote Terminal session. Mosh also has some quite cool features, that users of other modern shells will appreciate, such as auto complete of common commands and also a nifty indicator bar to tell you if something is up with the connection.

All in all, give mosh a go, it took me less than 120 seconds to get up and running. If you hate dead ssh sessions and can’t wait for timeouts, then what are you waiting for ;). As usual, hit me up at @italoarmstrong on twitter if you want banter.

 

Overclocking & Optimising the Raspberry PI

My Raspberry PI

Out the box the Raspberry PI comes with a ARM1176JZFS Core (armv6 with hard float aka armhf arch) running at 700 Mhz as part of the Broadcom SoC. Additionally the memory frequency is also limited. In recent firmwares however… tinkerers have had the ability to “overclock” the Raspberry PI to squeeze some extra juice out of it. Mine’s currently running at 1Ghz at a solid 48C temperature when under load. So the first question that springs to mind is… why doesn’t everyone overclock their Raspberry PI? Well… there have been (well founded) reports of SD card corruption, heat/power issues and instability. The idea of this post is to show the user how to safety squeeze every last bit, cycle and IOP out of their PI safely’ish and without being an astrophysicist. Read on for the know-how. … 

 

Installing Linux on a Late 2012 Mac Mini

So in the last post I discussed why the Mac Mini is the perfect machine for Linux and for Datacenters in general! One frustration some readers may be finding is that the networking chipset used by Ivy Bridge platform in Late 2012 Mac Mini’s doesn’t have native support in the Linux Kernel (as of now anyway). So its required to install a kernel module from the manufacturer/vendor (broadcom).

On their website they provide the “tg3” drivers for Linux kernels, however these are only good if you are running a Linux kernel < 3.5.x. If you take Ubuntu for example, 12.04 uses the 3.2.x stream, whereas 12.10 uses the 3.5.x stream and isn’t immediately compatible with the drivers on the broadcom page. This is due to the deprecation in 3.x and removal in 3.5.x of the asm/system.h header.

Read on for the fix, more and downloads. …