Xen on NetBSD x86 64

note: I'm pasting this from wiki.netbsd.se since it has already shut down.

I'm trying to install Xen hypervisor on our amd64 box with NetBSD-4.0. I'm getting a couple of errors on getting it to run. The friendly folks at #netbsd told me that I either need to use i386 kernel or use the bleeding edge NetBSD-CURRENT. They suggested that I setup a wiki page to document my progress.

Well, this is this page. I'll update this from time to time.

Switching from 4.0 to CURRENT
My box is currently using NetBSD 4.0:

root@:~# uname -v NetBSD 4.0 (GENERIC) #0: Sat Dec 15 22:25:31 PST 2007

I need to change my kernel to CURRENT. First, I need to fetch the sources as outlined in http://www.netbsd.org/docs/guide/en/chap-fetch.html:

(more to come... blah, blah, blah...)

Unorganized Notes
This section will be organized later.

Whoa... /usr/src/build.sh is very cool. Rebuilding the kernel is very easy. I just downloaded the sources, and executed ./build.sh tools kernel=RAZOR-XEN3_DOM, and that's it.

grub doesn't compile in amd64 via pkgsrc. 32-bit binary compatibility is required.

IMPORTANT: should be emulators/netbsd32_compat30, not emulators/netbsd32_compat40

just do pkg_add -f

It's a bit hard finding a decent FTP mirror for downloading the packages...

Problem executing grub-install after I did pkg_add:

root@razor:~# grub-install "(hd0)" /usr/pkg/sbin/grub: 1: Syntax error: "(" unexpected sed: /grub/device.map: No such file or directory grep: /grub/device.map: No such file or directory /dev/wd0a does not have any corresponding BIOS drive.

Reason: Some files are missing. /grub is empty.

It seems that neither emulators/netbsd32_compat30 or emulators/netbsd32_40 work:

root@razor:~# grub /usr/pkg/sbin/grub: Exec format error. Binary file not executable.

root@razor:~# file `which grub` /usr/pkg/sbin/grub: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for NetBSD 4.0, dynamically linked (uses shared libs), stripped

Blah. In addition to netbsd32_compat30, I also needed to install netbsd32_compat16 as well. Problem solved. grub now runs. Note that the correct grub package is the one that is built for NetBSD 3.1.

Problem solved, now I have a new problem:

root@razor:~# grub-install /dev/wd0d The file /grub/stage1 not read correctly.

Blah blah blah. It seems that grub doesn't recognize a root file system greater than 512MB. Mine is around 15GB. Waaaah...

Problem solved. In the grub shell, I just did:

install= (hd0,0,a)/stage1 (hd0,0,a) (hd0,0,a)/stage2 p (hd0,0,a)/menu.lst

Instead of the usual "setup (hd0)".

I can't boot. grub can't seem to read either /netbsd or /xen.gz. I'll just download the bootonly ISO and install CURRENT. I'll also make sure that the size for / is 512MB only.

w00t! I have now a running NetBSD dom0 on our amd64 box. I'll now reorganize this wiki page. To recap:


 * use i386 grub package built for NetBSD 3.1
 * compiling xenkernel3 via current pkgsrc, as of Feb 14, 2008, doesn't work. doing a pkg_add -f on a package that was built for NetBSD 4.0 works just fine though
 * / must be less than or equal to 512MB. My previous setup, which uses 15GB, didn't work. grub seems to have trouble reading large / partitions.
 * use NetBSD-CURRENT instead of NetBSD 4.0

root@razor:~# uname -a NetBSD razor.simoncpu.com 4.99.54 NetBSD 4.99.54 (XEN3_DOM0) #0: Sun Feb 10 03:24:57 PST 2008 builds@wb28:/home/builds/ab/HEAD/amd64/200802100002Z-obj/home/builds/ab/HEAD/src/sys/arch/amd64/compile/XEN3_DOM0 amd64

domU
rc.conf:

xend=YES xenbackendd=YES

brctl: not found

needs bridge-utils from pkgsrc-wip