GSoC 2017 Project: Integration of the OpenRISC Linux Port into OpTiMSoC

Linux was ported to OpTiMSoC during the 2017’s Google Sumer of Code. This blog post details the work that was accomplished during the project; and as well the work that was left to be tackled.

To port Linux to OpTiMSoC, a new manycore configuration with a Host Tile was designed and implemented. The Host Tile runs OpenRISC Linux and communicates with applications running on Computing Tiles via message-passing through the Network-on-Chip (NoC). Overall, the communication infrastructure is exported to user-level through standard UNIX file system operations, thereby enabling user-level applications running on the Host Tile to rely on a high-level communication abstractions.

I. Design

Figure 1 presents and the design overview of this project. A manycore configuration featuring six tiles is depicted, one host tile and five compute tiles. The host tile has a UART device attached to it and runs the OpenRISC port of Linux; whereas compute tiles runs user-level applications bare metal. Processes running on the Host Tile and user-level applications running on the Compute Tile communicate with one another via message-passing.

Figure 1: Overview of the target manycore configuration.

The OpenRisc Linux kernel running on the Host Tile features a Network-on-Chip (NoC) driver on which processes rely to send/receive messages. Operations on this driver are exported to userland through the standard UNIX system calls for manipulating files. Code Snippet 1 illustrates how a process running on the Host Tile send messages based on this abstraction. As a side remark, note that the user-level application should explicitly build the message header. In future implementation, the idea is to have a user-level library that does this job for the application. On the other hand, in Compute Tiles, user-level applications rely on a baremetal message-passing library to carry out a communication with other tiles.

int main(int argc, char **argv)
{
    int fd;
    const char *devname = "/dev/noc";

    /* Open NoC device. */
    fd = open(devname, O_WRONLY);
    
    /* Write some data. */
    while (int i = 0; i < 100; i++) {
        unsigned dest = 0x8000000;
        unsigned data = 0x0000dead;
        unsigned msg = dest | data;
        write(fd, &msg, sizeof(unsigned));
    }
   
    /* Close NoC device. */
    close(fd);

    return (EXIT_SUCCESS);
}

II. NoC Driver Implementation

To enable inter-tile communication, a NoC device driver for Linux was written. This device driver was implemented as a dynamically loadable module and it exports operations on the NoC to userland through regular file system calls. The main internal routines of the implemented driver are discussed next.

/*
 * Opens a NoC endpoint. The device minor number is used to
 * identify the target endpoint to open. Access to an endpoint
 * is exclusive, ie. only one process may use a given endpoint
 * at a time.
 */
int optimsoc_open(struct inode *, struct file *);

/*
 * Releases a NoC endpoint. The device minor number is is used
 * to identify the target  endpoint to release. Underlying
 * resources associated with the endpoint are released.
 */
int optimsoc_release(struct inode, struct file *);

/*
 * Sends messages over an endpoint. The device minor number
 * is used to identify the target endpoint to use on the
 * communication. Sending messages does not block the
 * calling process. The size of messages should be aligned
 * on word size (32 bits). Message headers should be built
 * on user-space.
 */
ssize_t optimsoc_write(struct file *, const char *, size_t, off_t);

/*
 * Receives messages over an endpoint. The device minor
 * number is used to identify the target endpoint to use on
 * the communication. Sending messages blocks the calling
 * process. Flits of a message are buffered in kernel land.
 */
ssize_t optimsoc_read(struct file *, const char *, size_t, off_t);

III. Building the Project

Since the Linux source tree is not yet integrated into OpTiMSoC Project, you should follow the next steps to get it running on OpTiMSoC.

Part 1: Setup OpTiMSoC

Follow the upstream online instructions at: https://www.optimsoc.org/docs/master/user_guide/installation.html

In the end, set the OPTIMSOC environment variable to point to the installation location of OpTiMSoC.

Part 2: Get Development Tools

To build Linux, you will need a slightly diferent toolchain than the one used to build OpTiMSoC.

  • Musl Toolchain
  • Newlib Baremetal Toolchain
mkdir -p $HOME/toolchain

cd $HOME/toolchain

wget  https://github.com/openrisc/or1k-gcc/releases/download/or1k-5.4.0-20170218/or1k-elf-5.4.0-20170218.tar.bz2

tar -xjvf or1k-elf-5.4.0-20170218.tar.bz2

wget https://github.com/openrisc/or1k-gcc/releases/download/or1k-5.4.0-20170218/or1k-linux-musl-5.4.0-20170218.tar.bz2

tar -xjvf or1k-linux-musl-5.4.0-20170218.tar.bz2

wget https://github.com/openrisc/or1k-gcc/releases/download/or1k-5.4.0-20170218/or1k-linux-5.4.0-20170218.tar.bz2

tar -xjvf or1k-linux-5.4.0-20170218.tar.bz2

Part 3: Build Unit Tests (Optional)

You can test your Linux build in OpTiMSoC with a simple distributed application:

  • hello-linux: runs on a host tile on top of Linux and simply sends raw messages to a remote compute tile.
  • hello-baremetal: runs on a compute tile on baremental, and simply read messages that arrive at the local NoC adapater and print them on the screen.
export LIBS=$OPTIMSOC/soc/sw/lib/baremetal/libbaremetal.a

git clone https://github.com/optimsoc/linux-apps $HOME/linux-apps

cd $HOME/linux-apps/

export CC=$HOME/toolchain/or1k-linux-musl/bin/or1k-linux-musl-gcc


$CC hello-linux/hello.c -o hello-linux/hello

export CC=$HOME/toolchain/or1k-elf/bin/or1k-elf-gcc
export CFLAGS=”-I $OPTIMSOC/soc/sw/include/baremetal/”

$CC $CFLAGS hello-baremetal/hello.c -o hello-baremetal/hello $LIBS

Part 4: Build Linux

Setup toolchain.

export PATH=$PATH:$HOME/toolchain/or1k-linux/bin/
export ARCH=openrisc
export CROSS_COMPILE=or1k-linux-

(Optional) If you have built unit tests from Step 3, you should copy hello-linux to the the initramfs directory.

mkdir -p $HOME/linux/arch/openrisc/initramfs/
    cp $HOME/linux-apps/hello-linux/hello $HOME/linux/arch/openrisc/initramfs/

Clone and build Linux. Note that this is done in three steps: (i) build system configuration; (ii) build dynamic modules; and (iii) build the kernel.

git clone https://github.com/optimsoc/linux.git $HOME/linux

cd $HOME/linux

make optimsoc_defconfig
make modules
make

III. Conclusions

During this GSoC project I wrote a Linux NoC driver for OpTiMSoC-based platforms. This driver was implemented as a dynamically loadable module and used standard file system kernel calls to export the message passing of the underlying platform to user space. Concerning the current implementation, the following work remains open:

  • [Enhancement] Enable Endpoints to Be Multiplexed. In the current implementation, only process at a time may use a given endpoint. A nice feature to have would be to enable multiple processes to concurrently use an endpoint.
  • [Bug Fix] Properly Install the Interrupt Handler. In the current implementation, when the interrupt handler is registered, the boot sequence halts when running on Verilator/FPGA. Interrupt-firing behavior should be investigated and the interrupt handler setup should be fixed accordingly.

by Pedro H. Penna
on 04 September 2017

Share this post at: