轉自:https://jgsun.github.io/2018/11/30/the-summary-of-running-arm-QEMU/
概述
這篇文章是用QEMU模擬運行arm u-boot和linux的一個總結,以arm vexpres板爲例,包括用QEMU單獨運行u-boot或者linux;實現QEMU運行u-boot和宿主機ubuntu網絡通信,u-boot用tftp下載方式引導linux;u-boot從SD卡或者flash引導linux。本文使用buildroot作爲構建系統,極大簡化了編譯和文件系統製作方面的工作。以buildroot自帶的configs/qemu_arm_vexpress_defconfig作爲起點,增加u-boot,改rootfs爲initramfs等,產生了本文所用的默認配置configs/qemu_arm_vexpress-fun_defconfig,保存在github網站:https://github.com/jgsun/buildroot,用於製作sd卡和flash image的腳本文件也放到了該github網站(board/qemu/scripts目錄)。
關鍵字:windows7,virtualbox,ubuntu18.04,buildroot
使用buildroot編譯
buildroot已經有arm vexpress的default配置,開始/repo/buildroot$ make qemu_arm_vexpress_defconfig
,然後在此基礎上在bootloader選擇u-boot,在toolchain選擇glibc,編譯即可,得到output/build/uboot-2018.05/u-boot,output/images/rootfs.ext2,output/images/zImage和output/images/vexpress-v2p-ca9.dtb用於QEMU啓動u-boot和linux。
QEMU運行u-boot
jgsun@jgsun-VirtualBox:/repo/buildroot$ qemu-system-arm -M vexpress-a9 -kernel output/build/uboot-2018.05/u-boot -nographic
pulseaudio: set_sink_input_volume() failed
pulseaudio: Reason: Invalid argument
pulseaudio: set_sink_input_mute() failed
pulseaudio: Reason: Invalid argument
U-Boot 2018.05 (Jul 08 2018 - 15:27:56 +0800)
DRAM: 128 MiB
WARNING: Caches not enabled
Flash: 128 MiB
QEMU運行linux
jgsun@jgsun-VirtualBox:/repo/buildroot$ cat board/qemu/arm-vexpress/readme.txt
Run the emulation with:
qemu-system-arm -M vexpress-a9 -smp 1 -m 256 -kernel output/images/zImage -dtb output/images/vexpress-v2p-ca9.dtb -drive file=output/images/rootfs.ext2,if=sd,format=raw -append "console=ttyAMA0,115200 root=/dev/mmcblk0" -serial stdio -net nic,model=lan9118 -net user
The login prompt will appear in the terminal that started Qemu. The
graphical window is the framebuffer.
If you want to emulate more cores change "-smp 1" to "-smp 2" for
dual-core or even "smp -4" for a quad-core configuration.
Tested with QEMU 2.12.0
啓動log
jgsun@jgsun-VirtualBox:/repo/buildroot$ qemu-system-arm -M vexpress-a9 -smp 1 -m 256 -kernel output/images/zImage -dtb output/images/vexpress-v2p-ca9.dtb -drive file=output/images/rootfs.ext2,if=sd,format=raw -append "console=ttyAMA0,115200 root=/dev/mmcblk0" -serial stdio -net nic,model=lan9118 -net user
pulseaudio: set_sink_input_volume() failed
pulseaudio: Reason: Invalid argument
pulseaudio: set_sink_input_mute() failed
pulseaudio: Reason: Invalid argument
Booting Linux on physical CPU 0x0
Linux version 4.16.7 (jgsun@jgsun-VirtualBox) (gcc version 7.3.0 (Buildroot 2018.08-git-00596-gb99dbdfac9)) #1 SMP Sun Jul 8 15:36:11 CST 2018
CPU: ARMv7 Processor [410fc090] revision 0 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
OF: fdt: Machine model: V2P-CA9
改用initramfs之後的啓動命令log
cd /repo/training/qemu/vexpress-a9
qemu-system-arm -M vexpress-a9 -smp 4 -m 1024 -kernel uImage -dtb vexpress-v2p-ca9.dtb -append "console=ttyAMA0,115200 root=/dev/ram0" -net nic,model=lan9118 -net user -nographic
jgsun@VirtualBox:/repo/training/qemu/vexpress-a9$ qemu-system-arm -M vexpress-a9 -smp 4 -m 1024 -kernel uImage -dtb vexpress-v2p-ca9.dtb -append "console=ttyAMA0,115200 root=/dev/ram0" -net nic,model=lan9118 -net user -nographic
pulseaudio: set_sink_input_volume() failed
pulseaudio: Reason: Invalid argument
pulseaudio: set_sink_input_mute() failed
pulseaudio: Reason: Invalid argument
Booting Linux on physical CPU 0x0
Linux version 4.16.7 (jgsun@VirtualBox) (gcc version 7.3.0 (Buildroot 2018.08-git-00596-gb99dbdfac9-dirty)) #7 SMP Thu Jul 12 18:56:46 CST 2018
CPU: ARMv7 Processor [410fc090] revision 0 (ARMv7), cr=10c5387d
...
udhcpc: started, v1.28.4
udhcpc: sending discover
udhcpc: sending select for 10.0.2.15
udhcpc: lease of 10.0.2.15 obtained, lease time 86400
deleting routers
adding dns 10.0.2.3
OK
Welcome to Buildroot
buildroot login: root
QEMU運行u-boot引導linux
網絡引導tftp方式啓動Linux內核
自建網橋br0
1.創建qemu-ifup_br0、qemu-ifdown_br0腳本
qemu-ifup_br0
cat board/qemu/scripts/qemu-ifup_br0
#!/bin/sh
run_cmd()
{
echo \$1
eval \$1
}
run_cmd "sudo ifconfig"
run_cmd "sudo brctl addbr br0"
run_cmd "sudo ifconfig br0 10.0.2.16 up"
run_cmd "sudo ip route"
run_cmd "sudo ip addr del 10.0.2.16/24 dev enp0s3" # del ip on enp0s3
run_cmd "sudo ip route"
run_cmd "sudo brctl addif br0 enp0s3"
run_cmd "sudo route add default gw 10.0.2.2"
run_cmd "sudo ip route"
run_cmd "sudo tunctl -u \$(id -un) -t \$1"
run_cmd "sudo ifconfig $1 0.0.0.0 promisc up"
run_cmd "sudo brctl addif br0 \$1"
run_cmd "brctl show"
qemu-ifdown_br0
cat board/qemu/scripts/qemu-ifdown_br0
#!/bin/sh
run_cmd()
{
echo \$1
eval \$1
}
run_cmd "sudo brctl delif br0 enp0s3"
run_cmd "sudo ifconfig br0 down"
run_cmd "sudo brctl delbr br0"
run_cmd "sudo ifconfig enp0s3 10.0.2.16 up"
run_cmd "brctl show"
run_cmd "sudo route add default gw 10.0.2.2"
run_cmd "ip route"
qemu-ifup_br0將作爲QEMU的net啓動script,創建TAP類型的接口tap0和網橋br0;添加tap0和enp0s3到br0;刪除enp0s3的ip,爲br0配置ip 10.0.2.15;這樣QEMU虛擬機u-boot和linux就能聯網:既能與宿主機ubuntu進行通信,也能聯通外網(如ping 服務器135.251.205.177)。
QEMU虛擬機u-boot引導啓動的linux會自動獲取和br0相同網段的IP 10.0.2.17,並添加了default route 10.0.2.2,如下所示:
# ip addr show dev eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 52:54:00:12:34:56 brd ff:ff:ff:ff:ff:ff
inet 10.0.2.17/24 brd 10.0.2.255 scope global eth0
valid_lft forever preferred_lft forever
# ip route
default via 10.0.2.2 dev eth0
10.0.2.0/24 dev eth0 scope link src 10.0.2.17
# ping 172.24.208.168
PING 172.24.208.168 (172.24.208.168): 56 data bytes
64 bytes from 172.24.208.168: seq=1 ttl=121 time=3.659 ms
64 bytes from 172.24.208.168: seq=2 ttl=121 time=2.787 ms
這種配置會讓宿主機ubuntu不能聯通外網(因爲其網卡enp0s3掛載到br0了),需要在宿主機添加default route gw 10.0.2.2之後才能聯通外網: sudo route add -net 0.0.0.0/32 gw 10.0.2.2 or sudo route add default gw 10.0.2.2
jgsun@VirtualBox:/repo/work/buildroot$ ip route
default via 10.0.2.2 dev br0
10.0.0.0/8 dev br0 proto kernel scope link src 10.0.2.15
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1
jgsun@VirtualBox:/repo/work/buildroot$ ping 172.24.208.168
PING 172.24.208.168 (172.24.208.168) 56(84) bytes of data.
64 bytes from 172.24.208.168: icmp_seq=1 ttl=121 time=1.63 ms
64 bytes from 172.24.208.168: icmp_seq=2 ttl=121 time=1.86 ms
如果QEMU虛擬機只需與宿主機通信,不需要聯通外網,可以不連接網卡enp0s3到網橋br0,就不影響宿主機連外網了,這時要手動將br0和QEMU虛擬機linux的網卡的ip配成一個網段如192.168.2.x。 2.帶net參數啓動u-boot,tap類型,名稱爲tap0,script爲步驟2創建的qemu-ifup_br0
jgsun@VirtualBox:/repo/buildroot$ sudo qemu-system-arm -M vexpress-a9 -m 1024 -kernel output/build/uboot-2018.05/u-boot -nographic -net nic,model=lan9118 -net tap,ifname=tap0,script=board/qemu/arm-vexpress/qemu-ifup_br0
[sudo] password for jgsun:
sudo tunctl -u root -t tap0
TUNSETIFF: Device or resource busy
sudo ifconfig tap0 0.0.0.0 promisc up
sudo brctl addif br0 tap0
brctl show
bridge name bridge id STP enabled interfaces
br0 8000.080027f6cde5 no enp0s3
tap0
vnet0
virbr0 8000.5254005634be yes virbr0-nic
pulseaudio: set_sink_input_volume() failed
pulseaudio: Reason: Invalid argument
pulseaudio: set_sink_input_mute() failed
pulseaudio: Reason: Invalid argument
U-Boot 2018.05 (Jul 08 2018 - 15:27:56 +0800)
DRAM: 128 MiB
WARNING: Caches not enabled
Flash: 128 MiB
MMC: MMC: 0
*** Warning - bad CRC, using default environment
In: serial
Out: serial
Err: serial
Net: smc911x-0
Hit any key to stop autoboot: 0
=>
3.設置環境變量ipaddr爲10.0.2.222,serverip爲10.0.2.15(ubuntu br0的ip),並ping通serverip
=> setenv ipaddr 10.0.2.222
=> setenv serverip 10.0.2.15
=> ping 10.0.2.15
smc911x: MAC 52:54:00:12:34:56
smc911x: detected LAN9118 controller
smc911x: phy initialized
smc911x: MAC 52:54:00:12:34:56
Using smc911x-0 device
smc911x: MAC 52:54:00:12:34:56
host 10.0.2.15 is alive
4.buildroot設置編譯 kernel裏面選擇kernel/Kernel binary format爲uImage;load address爲0x60003000; Filesystem images裏面選擇initramfs:
編譯得到image:uImage和vexpress-v2p-ca9.dtb拷貝到ubuntu的tftp server目錄/var/lib/tftpboot 5.在u-boot命令行修改/設置u-boot環境變量bootargs,並用tftp命令下載uImage和vexpress-v2p-ca9.dtb;然後用bootm命令啓動linux 注:修改u-boot源碼include/configs/vexpress_common.h,可以實現讓u-boot自動啓動linux #define CONFIG_BOOTCOMMAND "tftp 0x60003000 uImage; setenv bootargs 'root=/dev/mmcblk0 console=ttyAMA0';bootm 0x60003000"
setenv bootargs 'root=/dev/ram0 console=ttyAMA0'
tftp 0x60003000 uImage
tftp 0x60900000 vexpress-v2p-ca9.dtb
bootm 0x60003000 - 60900000
linux啓動界面:
=> bootm 0x60003000 - 60900000
## Booting kernel from Legacy Image at 60003000 ...
Image Name: Linux-4.16.7
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 6844432 Bytes = 6.5 MiB
Load Address: 60003000
Entry Point: 60003000
Verifying Checksum ... OK
## Flattened Device Tree blob at 60900000
Booting using the fdt blob at 0x60900000
Loading Kernel Image ... OK
Loading Device Tree to 7fee7000, end 7feed7ed ... OK
Starting kernel ...
Booting Linux on physical CPU 0x0
Linux version 4.16.7 (jgsun@VirtualBox) (gcc version 7.3.0 (Buildroot 2018.08-git-00596-gb99dbdfac9-dirty)) #6 SMP Wed Jul 11 15:43:21 CST 2018
CPU: ARMv7 Processor [410fc090] revision 0 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
OF: fdt: Machine model: V2P-CA9
Memory policy: Data cache writeback
CPU: All CPU(s) started in SVC mode.
使用qemu-kvm創建的bridge virtbr0
前面步驟1,2提到將宿主局網卡enp0s3加到網橋br0上面,宿主機ubuntu需要添加default route gw 10.0.2.2才能上外網。其實這裏還有一個方法:將tap0掛載到qemu-kvm創建的網橋vitbr0上面,能實現讓QEMU虛擬機與宿主機通信和聯通外網,也不影響宿主機聯通外網。 virbr0 是 KVM 默認創建的一個 bridge,其作用是爲連接其上的虛機網卡提供 NAT 訪問外網的功能。virbr0 默認分配了一個IP 192.168.122.1,併爲連接其上的其他虛擬網卡提供 DHCP 服務。 QEMU啓動u-boot的命令:sudo qemu-system-arm -M vexpress-a9 -m 1024 -kernel output/build/uboot-2018.05/u-boot -nographic -net nic,model=lan9118 -net tap,ifname=tap0,script=board/qemu/arm-vexpress/qemu-ifup
網絡啓動腳本是qemu-ifup:
jgsun@VirtualBox:/repo/work/buildroot$ cat board/qemu/arm-vexpress/qemu-ifup
#!/bin/sh
cmd="sudo tunctl -u $(id -un) -t $1"
echo ${cmd}
eval ${cmd}
cmd="sudo ifconfig $1 0.0.0.0 promisc up"
echo ${cmd}
eval ${cmd}
cmd="sudo brctl addif virbr0 $1"
echo ${cmd}
eval ${cmd}
cmd="brctl show"
echo ${cmd}
eval ${cmd}
QEMU虛擬機linux啓動之後自動獲取ip 192.168.122.76,將192.168.122.1作爲default route,能ping通我司主頁172.24.208.168:
# ip addr show dev eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 52:54:00:12:34:56 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.76/24 brd 192.168.122.255 scope global eth0
valid_lft forever preferred_lft forever
# ip route
default via 192.168.122.1 dev eth0
192.168.122.0/24 dev eth0 scope link src 192.168.122.76
# ping 172.24.208.168
PING 172.24.208.168 (172.24.208.168): 56 data bytes
64 bytes from 172.24.208.168: seq=0 ttl=120 time=7.689 ms
啓動log看出啓動udhcpc獲取ip
udhcpc: started, v1.28.4
udhcpc: sending discover
udhcpc: sending discover
udhcpc: sending select for 192.168.122.76
udhcpc: lease of 192.168.122.76 obtained, lease time 3600
deleting routers
adding dns 192.168.122.1
OK
Welcome to Buildroot
buildroot login: root
宿主機ubuntu的網絡設置:
jgsun@VirtualBox:/repo/work/buildroot$ brctl show
bridge name bridge id STP enabled interfaces
virbr0 8000.3a1064dd8595 yes tap0
jgsun@VirtualBox:~$ ip route
default via 10.0.2.2 dev enp0s3 proto dhcp metric 20100
10.0.2.0/24 dev enp0s3 proto kernel scope link src 10.0.2.16 metric 100
169.254.0.0/16 dev enp0s3 scope link metric 1000
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown
在QEMU虛擬機linux ping服務器135.251.205.177,tcpdump抓取enp0s3網口報文保存爲pcap文件用wireshark打開,可以看到源地址已經被替換成enp0s3的ip 10.0.2.15 /repo/work/buildroot$ sudo tcpdump -i enp0s3 -w /media/sf_tftpd/enp0s3.pcap
掛載NFS文件系統
1.ubuntu按照nfs server
sudo apt-get install nfs-kernel-server
sudo mkdir -p /root/rootfs
sudo vi /etc/exports//最後一行添加"/home/jgsun/rootfs *(sync,rw,no_root_squash)"
jgsun@VirtualBox:~$ sudo /etc/init.d/nfs-kernel-server restart
[ ok ] Restarting nfs-kernel-server (via systemctl): nfs-kernel-server.service.
2.QEMU虛擬機linux掛載nfs mount -t nfs -o nolock 192.168.122.1:/home/jgsun/work/rootfs /mnt/rootfs
SD卡啓動
1.製作一個8M大小的SD卡文件,並將啓動文件uImage和dtb拷貝到SD卡
dd if=/dev/zero of=sd.ext2 bs=1M count=8
mkfs.ext2 sd.ext2
mkdir tmpfs
sudo mount -t ext2 sd.ext2 tmpfs/ -o loop
sudo cp uImage vexpress-v2p-ca9.dtb tmpfs //將用uImage和dtb拷到此處
sudo umount tmpfs
2.QEMU帶-sd參數啓動u-boot sudo qemu-system-arm -M vexpress-a9 -m 1024 -kernel output/build/uboot-2018.05/u-boot -nographic -sd output/image/sd.ext2
ext2ls 命令顯示SD卡內文件內容:
=> ext2ls mmc 0
<DIR> 1024 .
<DIR> 1024 ..
<DIR> 12288 lost+found
6891184 uImage
14318 vexpress-v2p-ca9.dtb
3.ext2load命令從SD卡加載uImage和dtb到內存,設置bootargs之後用bootm啓動linux:
=> ext2load mmc 0 0x60003000 uImage
6891184 bytes read in 1188 ms (5.5 MiB/s)
=> ext2load mmc 0 0x60900000 vexpress-v2p-ca9.dtb
14318 bytes read in 59 ms (236.3 KiB/s)
=> setenv bootargs 'root=/dev/ram0 console=ttyAMA0'
=> bootm 0x60003000 - 60900000
## Booting kernel from Legacy Image at 60003000 ...
Image Name: Linux-4.16.7
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 6891120 Bytes = 6.6 MiB
Load Address: 60003000
Entry Point: 60003000
Verifying Checksum ... OK
## Flattened Device Tree blob at 60900000
nor flash啓動
1.製作一個8M大小的flash文件,燒寫uImage和dtb文件到flash,注意bs須是4096
dd if=/dev/zero of=flash.bin bs=4096 count=2048//sector size 4K,count 2K
dd if=vexpress-v2p-ca9.dtb of=flash.bin conv=notrunc bs=4096//copy dtb at the beginning
dd if=uImage of=flash.bin conv=notrunc bs=4096 seek=256//copy uImage at 1M from the beginning
2.QEMU帶-pflash參數啓動u-boot
sudo qemu-system-arm -M vexpress-a9 -m 1024 -kernel output/build/uboot-2018.05/u-boot -nographic -pflash output/image/flash.bin
3.cp從flash拷貝linux和dtb到內存,設置bootargs之後用bootm啓動linux:
=> cp.b 0x0 0x60900000 0x100000
=> cp.b 0x100000 0x60003000 0x700000
=> setenv bootargs 'root=/dev/ram0 console=ttyAMA0'
=> bootm 0x60003000 - 60900000
## Booting kernel from Legacy Image at 60003000 ...
Image Name: Linux-4.16.7
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 6891120 Bytes = 6.6 MiB
Load Address: 60003000
Entry Point: 60003000
Verifying Checksum ... OK
## Flattened Device Tree blob at 60900000
參考資料
- 用Qemu模擬vexpress-a9 — 配置 qemu 的網絡功能
- 用Qemu模擬vexpress-a9 (五) — u-boot引導kernel,device tree的使用
- ubuntu下使用qemu模擬ARM(七)—–uboot從sd卡啓動內核
- Booting Linux with U-Boot on QEMU ARM