dotnet

About .NET

http://mattwarren.org/2017/03/23/Hitchhikers-Guide-to-the-CoreCLR-Source-Code/

dotnet for x86/arm, ubuntu xenial, Work In Progress

Environment

export NETCORE=$HOME/w/netcore
export ROOTFS_X86=$HOME/w/rootfs/x86-xenial
# or
export ROOTFS_ARM=$HOME/w/rootfs/arm-xenial

Packages

sudo apt-get install lib32stdc++6
sudp apt-get install libxml2-utils

RootFS with CoreFX

sudo ROOTFS_DIR=$ROOTFS_X86 ./cross/build-rootfs.sh x86 xenial lldb3.8
sudo ROOTFS_DIR=$ROOTFS_ARM ./cross/build-rootfs.sh arm xenial lldb3.8

and maybe

choot $ROOTFS_X86 apt-get -y install \
gettext libcurl4-openssl-dev libicu-dev libkrb5-dev libssl-dev \
libunwind8-dev zlib1g-dev

CoreCLR

./build.sh clang3.9 skipnuget

ROOTFS_DIR=$ROOTFS_X86 ./build.sh x86 debug cross clang3.8 skipnuget

ROOTFS_DIR=$ROOTFS_ARM ./build.sh arm debug cross clang3.9 \
skipnuget cmakeargs "-DFEATURE_GDBJIT=TRUE -DSKIP_LLDBPLUGIN=TRUE"

CoreCLR unit test programs

# build in windows 10 host
cd netcore\coreclr
# build-test.cmd -rebuild -priority=2
build.cmd x86

Zip corclr/bin/tests/Windows_NT.x86.Debug into Windows_NT.x86.Debug.zip
Copy to $NETCORE and extract Zip
There should be $NETCORE/Windows_NT.x86.Debug folder

CoreFX

Cleaning

./build.sh -CleanAllProjects
git clean -fd

Build

# for host
sudo apt-get install symlinks zlib1g-dev libcurl4-openssl-dev libkrb5-dev libssl-dev
# build host and managed dlls
./build-native.sh -debug -- clang3.9

# for x86
ROOTFS_DIR=$ROOTF_X86 ./build-native.sh -debug -buildArch=x86 -- cross clang3.9

# for ARM
ROOTFS_DIR=$ROOTFS_ARM ./build-native.sh -debug -buildArch=arm -- cross clang3.9

# build managed (using linux-x64)
./build-managed.sh 

# build with test
./build-managed.sh -BuildTests=True -SkipTests=True -BuildPackages=False

# with Outerloop
./build-managed.sh -BuildTests=True -SkipTests=True -BuildPackages=False -- /p:Outerloop=True

CoreFX Testing

Prepare test overlay

mkdir -p $NETCORE/zoverlay.arm
cd  $NETCORE/zoverlay.arm

cp -v $NETCORE/corefx/bin/runtime/netcoreapp-Linux-Debug-x64/*.dll .
cp -v $NETCORE/corefx/bin/Linux.arm.Debug/native/*.so .
cp -vR $NETCORE/coreclr/bin/Product/Linux.arm.Debug/* .

# some test requires dotnet
cp corerun dotnet

# remove ni as it's x64
rm System.Private.CoreLib.ni.dll

# copy runner
cp -v $NETCORE/corefx/run-test.sh .

Test programs (Optional)

mkdir -p $NETCORE/zoverlay.arm/bin
cd $NETCORE/zoverlay.arm/bin

mkdir AnyOS.AnyCPU.Debug
mkdir Unix.AnyCPU.Debug
mkdir Linux.AnyCPU.Debug

cp -Rv $NETCORE/corefx/bin/AnyOS.AnyCPU.Debug/*.Tests AnyOS.AnyCPU.Debug/.
cp -Rv $NETCORE/corefx/bin/Unix.AnyCPU.Debug/*.Tests Unix.AnyCPU.Debug/.
cp -Rv $NETCORE/corefx/bin/Linux.AnyCPU.Debug/*.Tests Linux.AnyCPU.Debug/.

Invariant needs an option

vi AnyOS.AnyCPU.Debug/Invariant.Tests/netcoreapp/RunTests.sh
# add CORECLR_GLOBAL_INVARIANT=1
CORECLR_GLOBAL_INVARIANT=1 $RUNTIME_PATH/dotnet xunit.console.netcore.exe Invariant.Tests.dll ..

Run test

# in RPi2/3
cd $NETCORE/zoverlay.arm

$NETCORE/corefx/run-test.sh --runtime $NETCORE/zoverlay.arm \
--corefx-tests $NETCORE/zoverlay.arm/bin \
--os Linux --arch arm \
--sequential \
| tee $NETCORE/corefxtest-arm.log

You can use --corefx-tests $NETCORE/corefx/bin and don't copy the test programs.

A single test use --test-dir

$NETCORE/corefx/run-test.sh --runtime $NETCORE/zoverlay.arm \
--corefx-tests $NETCORE/zoverlay.arm/bin \
--os Linux --arch arm \
--sequential \
--test-dir Unix.AnyCPU.Debug/System.Net.WebSockets.Client.Tests \
| tee $NETCORE/corefxtest-rpi3.log

Check with direct connection

System.Net.WebSockets.Client.Tests
System.Net.Http.Functional.Tests
System.Net.Tests --> X
System.Net.Requests.Tests

dotnet for ARM with LLVM-3.8 on ARM Ubuntu Mate 16.04 Xenial

(Follow https://github.com/seanshpark/help/wiki/LLDB to prepare LLVM-3.8)

Environment

export NETCORE=$HOME/v/netcore

Prepare ARM root fs

Same as it from above LLDB link

cd $NETCORE/coreclr
sudo ROOTFS_DIR=$HOME/v/rootfs/arm-xenial ./cross/build-rootfs.sh arm xenial

NET Core build scripts

cd $NETCORE
git clone git@github.com:seanshpark/netcore_bin.git

CoreCLR / CoreFX for ARM with netcore_bin

cd $NETCORE

BASE_PATH=$NETCORE \
ROOTFS_DIR=$HOME/v/rootfs/arm-xenial LLVM_ARM_HOME=$HOME/v/bin/llvm-3.8-arm \
./netcore_bin/dotnet-build-arm.sh clean debug coreclr corefx \
--skip-build-tests --enable-jit-debug

CoreCLR build with default build.sh

cd coreclr
ROOTFS_DIR=$HOME/v/rootfs/arm-xenial ./build.sh arm debug cross skipnuget
# or with skiptests if want to also skip building tests

NET Core Overlay

BASE_PATH=$NETCORE \
./netcore_bin/build-overlay.sh Linux.arm.Debug \
--coreClrBinDir=coreclr/bin/Product/Linux.arm.Debug \
--coreFxBinDir="corefx/bin/Linux.AnyCPU.Debug;corefx/bin/Unix.AnyCPU.Debug;corefx/bin/AnyOS.AnyCPU.Debug" \
--coreFxNativeBinDir=corefx/bin/Linux.arm.Debug

export `v` to RPi2/3

/etc/exports

/home/maxwell/v 192.168.1.0/255.255.255.0(rw,sync,no_subtree_check,no_root_squash)

Prepare RPi2/3 Ubuntu Mate 16.04

download and flash

https://ubuntu-mate.org/raspberry-pi/

NFS mount in RPi2/3

sudo mount ubuntuhost:/maxwell/v $HOME/v

where ubuntuhost is Ubuntu 16.04 x86-64 build host

running in RPi2/3

cd $NETCORE/dotnet-overlay

./corerun ConsoleApp1.dll

where ConsoleApp1.dll is a sample .NET Core hello world application

CoreCLR/CoreFX native build inside RPi3

CoreCLR

./build.sh clang3.8 skipnuget skiptests

CoreFX

./src/Native/build-native.sh arm clang3.8
./build.sh arm skipgenerateversion skipmscorlib clang3.8

OLD

CoreCLR for ARM with LLVM-3.8

# assume llvm-3.8 build from source is installed at
# /home/maxwell/xp_seanrpi2/llvm/install-38.arm/llvm

# build coreclr
ROOTFS_DIR=/home/maxwell/zarm-rootfs-wily \
LLVM_ARM_HOME=/home/maxwell/xp_seanrpi2/llvm/install-38.arm \
./build.sh arm cross skipmscorlib clang3.8

# copy to RPi2 NFS
cp -R bin/Product/Linux.arm.Debug/* /home/maxwell/xp_seanrpi2/dotnet.38/.

CoreFX for ARM with LLVM-3.8

# build native
ROOTFS_DIR=/home/maxwell/zarm-rootfs-wily \
LLDB_LIB_DIR=/home/maxwell/xp_seanrpi2/llvm/install-38.arm/lib \
LLDB_INCLUDE_DIR=/home/maxwell/xp_seanrpi2/llvm/install-38.arm/include \
./build-native.sh -SkipTests=True -buildArch=arm -CleanAllProjects -- cross clang3.8

# build managed
./build-managed.sh -buildArch=arm -os=Linux -skiptests

using test overlay

Better way to to prepare DLL files seems to be using unit test overlay folder.

using copycorefx.sh script to prepare run environment

# copy to RPi2 NFS
../copycorefx.sh `pwd` Linux Debug /home/maxwell/xp_seanrpi2/dotnet.38

#!/usr/bin/env bash

usage()
{
    echo "Usage: $0 CoreFx BuildOS BuildType Destination"
    echo "CoreFX: corefx folder"
    echo "BuildOS can be: Windows_NT, Linux, OSX, FreeBSD"
    echo "BuildType can be: Debug, Release"
    echo "Destination is the folder where assemblies will be copied over"

    exit 1
}

copy_assemblies()
{
for file in `ls -1 $1`; do
    if [ -d $1/$file ]; then
        if [ -f $1/$file/$file.dll ]; then
            cp -vf $1/$file/$file.dll $2
        fi
    fi
done
}

if [ $# -lt 4 ]; then
    usage
    exit 1
else
    __CoreFXDir=$1
    __BuildOS=$2
    __BuildType=$3
    __Destination=$4
fi

echo "Building CoreFX $1 assemblies $2.$3 into $4"

#__CoreFXDir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )


echo ">>> " $__CoreFXDir/bin/AnyOS.AnyCPU.$__BuildType $__Destination
copy_assemblies $__CoreFXDir/bin/AnyOS.AnyCPU.$__BuildType $__Destination

echo ">>> " $__CoreFXDir/bin/Unix.AnyCPU.$__BuildType $__Destination
copy_assemblies $__CoreFXDir/bin/Unix.AnyCPU.$__BuildType $__Destination

echo ">>> " $__CoreFXDir/bin/$__BuildOS.AnyCPU.$__BuildType $__Destination
copy_assemblies $__CoreFXDir/bin/$__BuildOS.AnyCPU.$__BuildType $__Destination

echo ">>> do manual cp $__CoreFXDir/bin/$__BuildOS.(?).$__BuildType/Native/*.so $__Destination"

Debug CoreCLR with tests

build CoreCLR with tests also in Windows
Follow https://github.com/dotnet/coreclr/blob/master/Documentation/building/debugging-instructions.md#debugging-coreclr-on-windows to setup VS2015 for debugging
create a folder

copy all CoreCLR files in coreclr\bin\Product\Windows_NT.x64.Debug
copy all files in coreclr\bin\tests\Windows_NT.x64.Debug\Tests\Core_Root
copy unit test file want to debug

in VS2015, proj properties, set environment to 3) path/files

Inside CoreCLR

PAL SEH

SEH for Structured Exception Handling

This is what I've found in the source.

src/inc/palclr.h, Line 222, Here

EXCEPTION_RECORD

in pal.h

typedef struct _EXCEPTION_RECORD {
    DWORD ExceptionCode;
    DWORD ExceptionFlags;
    struct _EXCEPTION_RECORD *ExceptionRecord;
    PVOID ExceptionAddress;
    DWORD NumberParameters;
    ULONG_PTR ExceptionInformation[EXCEPTION_MAXIMUM_PARAMETERS];
} EXCEPTION_RECORD, *PEXCEPTION_RECORD;

in lldbservices.h

// This copy of the "64" bit record has been modified
// by removing the alignment field to make it the same
// as the _EXCEPTION_RECORD used in the pal defined in 
// pal.h.
typedef struct _EXCEPTION_RECORD64 {
    DWORD ExceptionCode;
    DWORD ExceptionFlags;
    DWORD64 ExceptionRecord;
    DWORD64 ExceptionAddress;
    DWORD NumberParameters;
    DWORD64 ExceptionInformation[EXCEPTION_MAXIMUM_PARAMETERS];
} EXCEPTION_RECORD64, *PEXCEPTION_RECORD64;

Is this safe for 32bit?

Old... for history

CoreCLR cross for arm-linux

qemu

https://github.com/dotnet/coreclr/blob/master/Documentation/building/cross-building.md

sudo apt-get install qemu qemu-user-static binfmt-support debootstrap

build coreclr

# build
ROOTFS_DIR=~/arm-rootfs-wily/ ./build.sh clean arm cross verbose skipmscorlib

build mscorlib in Windows

build.cmd linuxmscorlib arm

CoreFX cross for arm-linux

Build

# prepare arm-linux rootfs for corefx
sudo ROOTFS_DIR=~/arm-rootfs-corefx/ ./cross/build-rootfs.sh arm
ROOTFS_DIR=~/arm-rootfs-corefx/ ./build.sh clean arm verbose managed skiptests

Copy CoreFX DLLs with below script

../copycorefx.sh `pwd` Linux Debug (dest folder)

Copy System.Native.so file

cp ./bin/Linux.arm.Debug/Native/System.Native.so (dest folder)

mounting RPi image file: experimental... not finished.

download image file, ex) ubuntu-mate-15.10.3-desktop-armhf-raspberry-pi-2.img
get the root fs position

parted ubuntu-mate-15.10.3-desktop-armhf-raspberry-pi-2.img

unit B
p

Number  Start      End          Size         Type     File system  Flags
 1      1048576B   68157439B    67108864B    primary  fat16        boot, lba
 2      68157440B  3932159999B  3864002560B  primary  ext4

mount

sudo mount -o loop,offset=68157440 -t ext4 ubuntu-mate-15.10.3-desktop-armhf-raspberry-pi-2.img /mnt/partition

give ROOTFS_DIR= to the mount folder of RPi2 image

To do...

packages needs install, that may need to run in RPi2 or with chroot

debug coreclr

Turn on log

export PAL_DBG_CHANNELS="+all.all"

Turn off log

unset PAL_DBG_CHANNELS

etc

Check 1)

https://www.myget.org/F/dotnet-buildtools/FindPackagesById()?id='Microsoft.DotNet.BuildTools'

corefx/Tools/dotnetcli/bin/dotnet restore ./project.json \
--packages ./packages/ \
--source https://www.myget.org/F/dotnet-buildtools/

CI

http://dotnet-ci.cloudapp.net/job/dotnet_coreclr/job/master/

Home

dotnet

About .NET

dotnet for x86/arm, ubuntu xenial, Work In Progress

Environment

Packages

RootFS with CoreFX

CoreCLR

CoreCLR unit test programs

CoreFX

CoreFX Testing

dotnet for ARM with LLVM-3.8 on ARM Ubuntu Mate 16.04 Xenial

Environment

Prepare ARM root fs

NET Core build scripts

CoreCLR / CoreFX for ARM with netcore_bin

CoreCLR build with default build.sh

NET Core Overlay

export v to RPi2/3

Prepare RPi2/3 Ubuntu Mate 16.04

download and flash

NFS mount in RPi2/3

running in RPi2/3

CoreCLR/CoreFX native build inside RPi3

OLD

CoreCLR for ARM with LLVM-3.8

CoreFX for ARM with LLVM-3.8

using test overlay

using copycorefx.sh script to prepare run environment

Debug CoreCLR with tests

Inside CoreCLR

PAL SEH

EXCEPTION_RECORD

Old... for history

CoreCLR cross for arm-linux

CoreFX cross for arm-linux

mounting RPi image file: experimental... not finished.

debug coreclr

etc

CI

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

export `v` to RPi2/3