FORM：https://www.sohu.com/a/272455844_610730
英文原版FROM：https://opensource.com/article/18/10/kbuild-and-kconfig
深入瞭解Linux配置/構建系統的工作原理。
自從Linux內核代碼遷移到Git以來，Linux內核配置/構建系統（也稱爲Kconfig /kbuild）已經存在了很長時間。然而，作爲支撐基礎設施，它很少成爲人們關注的焦點；甚至在日常工作中使用它的內核開發人員也從未真正過它。
爲了探索如何編譯Linux內核，本文將深入研究Kconfig/kbuild內部過程，解釋如何生成.config文件和vmlinux/bzImage文件，並介紹依賴性跟蹤的智能技巧。
Kconfig
構建內核的第一步始終是Kconfig，Kconfig有助於使Linux內核高度模塊化和可定製。 Kconfig爲用戶提供了許多配置目標：
| config | 使用基於行的程序更新當前配置 |
| nconfig | 使用ncurses基於菜單的程序更新當前配置 |
| menuconfig | 使用基於菜單的程序更新當前配置 |
| xconfig | 使用基於Qt的前端更新當前配置 |
| gconfig | 使用基於GTK +的前端更新當前配置 |
| oldconfig | 使用提供的.config作爲基礎更新當前配置 |
| localmodconfig | 更新當前配置禁用未加載的模塊 |
| localyesconfig | 更新當前配置，將本地mod轉換到內核 |
| defconfig | 默認來自Arch提供的defconfig的新配置 |
| savedefconfig | 將當前配置保存爲./defconfig（最小配置） |
| allnoconfig | 新配置，其中所有選項均以“否”回答 |
| allyesconfig | 新配置，其中所有選項都被'yes'接受 |
| allmodconfig | 儘可能新配置選擇模塊 |
| alldefconfig | 新配置，所有符號都設置爲默認值 |
| randconfig | 新配置，隨機回答所有選項 |
| listnewconfig | 列出新選項 |
| olddefconfig | 與oldconfig相同，但在沒有提示的情況下將新符號設置爲其默認值 |
| kvmconfig | 爲KVM虛擬機內核支持啓用其他選項 |
| xenconfig | 爲xen dom0和虛擬機內核支持啓用其他選項 |
| tinyconfig | 配置最小的內核 |
我認爲menuconfig是這些選項中最受歡迎的。目標由不同的主程序處理，這些程序由內核提供並在內核構建期間構建。一些目標有一個GUI（爲了方便用戶），而大多數沒有。與Kconfig相關的工具和源代碼主要位於內核源代碼中的s/kconfig /下。正如我們從s/kconfig/Makefile中看到的，有幾個主機程序，包括conf，mconf和nconf。除了conf之外，它們中的每一個都負責基於GUI的配置目標之一，因此，conf處理它們中的大多數。
從邏輯上講，Kconfig的基礎結構有兩部分：一部分實現一種新語言來定義配置項（參見內核源代碼下的Kconfig文件），另一部分解析Kconfig語言並處理配置操作。
大多數配置目標具有大致相同的內部過程（如下所示）：
請注意，所有配置項都具有默認值。
第一步是讀取源根目錄下的Kconfig文件，構建初始配置數據庫; 然後它通過根據此優先級讀取現有配置文件來更新初始數據庫：
.config
/lib/modules/$(shell,uname -r)/.config
/etc/kernel-config
/boot/config-$(shell,uname -r)
ARCH_DEFCONFIG
arch/$(ARCH)/defconfig
如果你通過menuconfig進行基於GUI的配置或通過oldconfig進行基於命令行的配置，則會根據你的自定義更新數據庫。最後，將配置數據庫轉儲到.config文件中。
但.config文件不是內核構建的最終配置；這就是syncconfig目標存在的原因。syncconfig曾經是一個名爲silentoldconfig的配置選項，但它不會執行舊名稱所說的內容，因此它已重命名。此外，因爲它是供內部使用（不適用於用戶），所以它已從列表中刪除。
以下是syncconfig的作用：
syncconfig將.config作爲輸入並輸出許多其他文件，這些文件分爲三類：
- auto.conf和tristate.conf用於makefile文本處理。 例如，可以在組件的makefile中看到這樣的語句：
obj-$(CONFIG_GENERIC_CALIBRATE_DELAY) += calibrate.o
- autoconf.h用於C語言源文件。
- include / config /下的空頭文件用於kbuild期間的配置依賴性跟蹤，如下所述。
配置完成後，我們將知道哪些文件和代碼片段未編譯。
kbuild
組件式構建（稱爲遞歸make）是GNU make管理大型項目的常用方法。Kbuild是遞歸make的一個很好的例子。通過將源文件劃分爲不同的模塊/組件，每個組件都由其自己的makefile管理。當開始構建時，頂級makefile以正確的順序調用每個組件的makefile，構建組件，並將它們收集到最終的執行程序中。
Kbuild指的是不同類型的makefile：
- Makefile是位於源根目錄中的頂級makefile。
- .config是內核配置文件。
- arch / $（ARCH）/Makefile是arch makefile，它是top makefile的補充。
- s/Makefile。*描述了所有kbuild makefile的通用規則。
- 最後，大約有500個kbuild makefile。
top makefile包含arch makefile，讀取.config文件，下載到子目錄，在s/Makefile。*中定義的例程的幫助下，在每個組件的makefile上調用make，構建每個中間對象，並將所有中間對象鏈接到vmlinux中。內核文檔Documentation/kbuild/makefiles.txt描述了這些makefile的所有方面。
作爲示例，讓我們看看如何在x86-64上生成vmlinux：
（該插圖基於Richard Y. Steven的博客。它已更新，並在作者允許的情況下使用。）
進入vmlinux的所有.o文件首先進入他們自己的內置.a，通過變量KBUILD_VMLINUX_INIT，KBUILD_VMLINUX_MAIN，KBUILD_VMLINUX_LIBS指示，然後收集到vmlinux文件中。
在簡化的makefile代碼的幫助下，瞭解如何在Linux內核中實現遞歸make：
# In top Makefile
vmlinux: s/link-vmlinux.sh $(vmlinux-deps)
+$(call if_changed,link-vmlinux)
# Variable assignments
vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN) $(KBUILD_VMLINUX_LIBS)
export KBUILD_VMLINUX_INIT := $(head-y) $(init-y)
export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y2) $(drivers-y) $(net-y) $(virt-y)
export KBUILD_VMLINUX_LIBS := $(libs-y1)
export KBUILD_LDS := arch/$(SRCARCH)/kernel/vmlinux.lds
init-y := init/
drivers-y := drivers/ sound/ firmware/
net-y := net/
libs-y := lib/
core-y := usr/
virt-y := virt/
# Transform to corresponding built-in.a
init-y := $(patsubst %/, %/built-in.a, $(init-y))
core-y := $(patsubst %/, %/built-in.a, $(core-y))
drivers-y := $(patsubst %/, %/built-in.a, $(drivers-y))
net-y := $(patsubst %/, %/built-in.a, $(net-y))
libs-y1 := $(patsubst %/, %/lib.a, $(libs-y))
libs-y2 := $(patsubst %/, %/built-in.a, $(filter-out %.a, $(libs-y)))
virt-y := $(patsubst %/, %/built-in.a, $(virt-y))
# Setup the dependency. vmlinux-deps are all intermediate objects, vmlinux-dirs
# are phony targets, so every time comes to this rule, the recipe of vmlinux-dirs
# will be executed. Refer "4.6 Phony Targets" of `info make`
$(sort $(vmlinux-deps)): $(vmlinux-dirs) ;
# Variable vmlinux-dirs is the directory part of each built-in.a
vmlinux-dirs := $(patsubst %/,%,$(filter %/, $(init-y) $(init-m)
$(core-y) $(core-m) $(drivers-y) $(drivers-m)
$(net-y) $(net-m) $(libs-y) $(libs-m) $(virt-y)))
# The entry of recursive make
$(vmlinux-dirs):
$(Q)$(MAKE) $(build)=$@ need-builtin=1
例如，擴展了遞歸製作配方
make -f s/Makefile.build obj=init need-builtin=1
這意味着make將進入s/Makefile.build繼續構建每個內置的工作.a。 在s/link-vmlinux.sh的幫助下，vmlinux文件最終位於源根目錄下。
瞭解vmlinux與bzImage
許多Linux內核開發人員可能不清楚vmlinux和bzImage之間的關係。例如，這是他們在x86-64中的關係：
根vmlinux被剝離，壓縮，放入piggy.S，然後與其他對等對象鏈接到arch/x86/boot/compressed/vmlinux。同時，在arch / x86 / boot下生成一個名爲setup.bin的文件。可能存在具有重定位信息的可選第三文件，具體取決於CONFIG_X86_NEED_RELOCS的配置。
由內核提供的稱爲build的宿主程序將這兩個（或三個）部分構建到最終的bzImage文件中。
依賴性跟蹤
Kbuild跟蹤三種依賴關係：
第一個很容易理解，但第二個和第三個呢？ 內核開發人員經常會看到如下代碼：
#ifdef CONFIG_SMP
__boot_cpu_id = cpu;
#endif
當CONFIG_SMP更改時，應該重新編譯這段代碼。 編譯源文件的命令行也很重要，因爲不同的命令行可能會導致不同的目標文件。
當.c文件通過#include指令使用頭文件時，您需要編寫如下規則：
main.o: defs.h
recipe...
管理大型項目時，需要大量的這些規則；把它們全部寫下來會很乏味和乏味。幸運的是，大多數現代C編譯器都可以通過查看源文件中的#include行來編寫這些規則。對於GNU編譯器集合（GCC），只需添加命令行參數：-MD depfile
# In s/Makefile.lib
c_flags = -Wp,-MD,$(depfile) $(NOSTDINC_FLAGS) $(LINUXINCLUDE)
-include $(srctree)/include/linux/compiler_types.h
$(__c_flags) $(modkern_cflags)
$(basename_flags) $(modname_flags)
這將生成一個.d文件，內容如下：
init_task.o: init/init_task.c include/linux/kconfig.h
include/generated/autoconf.h include/linux/init_task.h
include/linux/rcupdate.h include/linux/types.h
...
然後主機程序fixdep通過將depfile和命令行作爲輸入來處理其他兩個依賴項，然後以makefile語法輸出。<target> .cmd文件，該文件記錄命令行和所有先決條件（包括配置） 爲目標。 它看起來像這樣：
# The command line used to compile the target
cmd_init/init_task.o := gcc -Wp,-MD,init/.init_task.o.d -nostdinc ...
...
# The dependency files
deps_init/init_task.o :=
$(wildcard include/config/posix/timers.h)
$(wildcard include/config/arch/task/struct/on/stack.h)
$(wildcard include/config/thread/info/in/task.h)
...
include/uapi/linux/types.h
arch/x86/include/uapi/asm/types.h
include/uapi/asm-generic/types.h
...
在遞歸make中將包含一個。<target> .cmd文件，提供所有依賴關係信息並幫助決定是否重建目標。
這背後的祕密是fixdep將解析depfile（.d文件），然後解析內部的所有依賴文件，搜索所有CONFIG_字符串的文本，將它們轉換爲相應的空頭文件，並將它們添加到目標的先決條件。每次配置更改時，相應的空頭文件也將更新，因此kbuild可以檢測到該更改並重建依賴於它的目標。 因爲還記錄了命令行，所以很容易比較最後和當前的編譯參數。
展望未來
Kconfig/kbuild在很長一段時間內保持不變，直到新的維護者Masahiro Yamada於2017年初加入，現在kbuild再次正在積極開發。 如果你很快就會看到與本文中的內容不同的內容，請不要感到驚訝。
英文原版
FROM：https://opensource.com/article/18/10/kbuild-and-kconfig
The Linux kernel config/build system, also known as Kconfig/kbuild, has been around for a long time, ever since the Linux kernel code migrated to Git. As supporting infrastructure, however, it is seldom in the spotlight; even kernel developers who use it in their daily work never really think about it.
To explore how the Linux kernel is compiled, this article will dive into the Kconfig/kbuild internal process, explain how the .config file and the vmlinux/bzImage files are produced, and introduce a smart trick for dependency tracking.
Kconfig
The first step in building a kernel is always configuration. Kconfig helps make the Linux kernel highly modular and customizable. Kconfig offers the user many config targets:
| config | Update current config utilizing a line-oriented program |
| nconfig | Update current config utilizing a ncurses menu-based program |
| menuconfig | Update current config utilizing a menu-based program |
| xconfig | Update current config utilizing a Qt-based frontend |
| gconfig | Update current config utilizing a GTK+ based frontend |
| oldconfig | Update current config utilizing a provided .config as base |
| localmodconfig | Update current config disabling modules not loaded |
| localyesconfig | Update current config converting local mods to core |
| defconfig | New config with default from Arch-supplied defconfig |
| savedefconfig | Save current config as ./defconfig (minimal config) |
| allnoconfig | New config where all options are answered with 'no' |
| allyesconfig | New config where all options are accepted with 'yes' |
| allmodconfig | New config selecting modules when possible |
| alldefconfig | New config with all symbols set to default |
| randconfig | New config with a random answer to all options |
| listnewconfig | List new options |
| olddefconfig | Same as oldconfig but sets new symbols to their default value without prompting |
| kvmconfig | Enable additional options for KVM guest kernel support |
| xenconfig | Enable additional options for xen dom0 and guest kernel support |
| tinyconfig | Configure the tiniest possible kernel |
I think menuconfig is the most popular of these targets. The targets are processed by different host programs, which are provided by the kernel and built during kernel building. Some targets have a GUI (for the user's convenience) while most don't. Kconfig-related tools and source code reside mainly under scripts/kconfig/ in the kernel source. As we can see from scripts/kconfig/Makefile, there are several host programs, including conf, mconf, and nconf. Except for conf, each of them is responsible for one of the GUI-based config targets, so, conf deals with most of them.
Logically, Kconfig's infrastructure has two parts: one implements a new language to define the configuration items (see the Kconfig files under the kernel source), and the other parses the Kconfig language and deals with configuration actions.
Most of the config targets have roughly the same internal process (shown below):
kconfig_process.png
The Linux Terminal
- Top 7 terminal emulators for Linux
- 10 command-line tools for data analysis in Linux
- Download Now: SSH cheat sheet
- Advanced Linux commands cheat sheet
- Linux command line tutorials
Note that all configuration items have a default value.
The first step reads the Kconfig file under source root to construct an initial configuration database; then it updates the initial database by reading an existing configuration file according to this priority:
.config
/lib/modules/$(shell,uname -r)/.config
/etc/kernel-config
/boot/config-$(shell,uname -r)
ARCH_DEFCONFIG
arch/$(ARCH)/defconfig
If you are doing GUI-based configuration via menuconfig or command-line-based configuration via oldconfig, the database is updated according to your customization. Finally, the configuration database is dumped into the .config file.
But the .config file is not the final fodder for kernel building; this is why the syncconfigtarget exists. syncconfig used to be a config target called silentoldconfig, but it doesn't do what the old name says, so it was renamed. Also, because it is for internal use (not for users), it was dropped from the list.
Here is an illustration of what syncconfig does:
syncconfig.png
syncconfig takes .config as input and outputs many other files, which fall into three categories:
- auto.conf & tristate.conf are used for makefile text processing. For example, you may see statements like this in a component's makefile:
obj-$(CONFIG_GENERIC_CALIBRATE_DELAY) += calibrate.o - autoconf.h is used in C-language source files.
- Empty header files under include/config/ are used for configuration-dependency tracking during kbuild, which is explained below.
After configuration, we will know which files and code pieces are not compiled.
kbuild
Component-wise building, called recursive make, is a common way for GNU make to manage a large project. Kbuild is a good example of recursive make. By dividing source files into different modules/components, each component is managed by its own makefile. When you start building, a top makefile invokes each component's makefile in the proper order, builds the components, and collects them into the final executive.
Kbuild refers to different kinds of makefiles:
- Makefile is the top makefile located in source root.
- .config is the kernel configuration file.
- arch/$(ARCH)/Makefile is the arch makefile, which is the supplement to the top makefile.
- scripts/Makefile.* describes common rules for all kbuild makefiles.
- Finally, there are about 500 kbuild makefiles.
The top makefile includes the arch makefile, reads the .config file, descends into subdirectories, invokes make on each component's makefile with the help of routines defined in scripts/Makefile.*, builds up each intermediate object, and links all the intermediate objects into vmlinux. Kernel document Documentation/kbuild/makefiles.txtdescribes all aspects of these makefiles.
As an example, let's look at how vmlinux is produced on x86-64:
vmlinux_generation_process.png
(The illustration is based on Richard Y. Steven's blog. It was updated and is used with the author's permission.)
All the .o files that go into vmlinux first go into their own built-in.a, which is indicated via variables KBUILD_VMLINUX_INIT, KBUILD_VMLINUX_MAIN, KBUILD_VMLINUX_LIBS, then are collected into the vmlinux file.
Take a look at how recursive make is implemented in the Linux kernel, with the help of simplified makefile code:
# In top Makefile vmlinux: scripts/link-vmlinux.sh $(vmlinux-deps) +$(call if_changed,link-vmlinux) # Variable assignments vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN) $(KBUILD_VMLINUX_LIBS) export KBUILD_VMLINUX_INIT := $(head-y) $(init-y) export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y2) $(drivers-y) $(net-y) $(virt-y) export KBUILD_VMLINUX_LIBS := $(libs-y1) export KBUILD_LDS := arch/$(SRCARCH)/kernel/vmlinux.lds init-y := init/ drivers-y := drivers/ sound/ firmware/ net-y := net/ libs-y := lib/ core-y := usr/ virt-y := virt/ # Transform to corresponding built-in.a init-y := $(patsubst %/, %/built-in.a, $(init-y)) core-y := $(patsubst %/, %/built-in.a, $(core-y)) drivers-y := $(patsubst %/, %/built-in.a, $(drivers-y)) net-y := $(patsubst %/, %/built-in.a, $(net-y)) libs-y1 := $(patsubst %/, %/lib.a, $(libs-y)) libs-y2 := $(patsubst %/, %/built-in.a, $(filter-out %.a, $(libs-y))) virt-y := $(patsubst %/, %/built-in.a, $(virt-y)) # Setup the dependency. vmlinux-deps are all intermediate objects, vmlinux-dirs # are phony targets, so every time comes to this rule, the recipe of vmlinux-dirs # will be executed. Refer "4.6 Phony Targets" of `info make` $(sort $(vmlinux-deps)): $(vmlinux-dirs) ; # Variable vmlinux-dirs is the directory part of each built-in.a vmlinux-dirs := $(patsubst %/,%,$(filter %/, $(init-y) $(init-m) \ $(core-y) $(core-m) $(drivers-y) $(drivers-m) \ $(net-y) $(net-m) $(libs-y) $(libs-m) $(virt-y))) # The entry of recursive make $(vmlinux-dirs): $(Q)$(MAKE) $(build)=$@ need-builtin=1
The recursive make recipe is expanded, for example:
make -f scripts/Makefile.build obj=init need-builtin=1
This means make will go into scripts/Makefile.build to continue the work of building each built-in.a. With the help of scripts/link-vmlinux.sh, the vmlinux file is finally under source root.
Understanding vmlinux vs. bzImage
Many Linux kernel developers may not be clear about the relationship between vmlinux and bzImage. For example, here is their relationship in x86-64:
vmlinux-bzimage.png
The source root vmlinux is stripped, compressed, put into piggy.S, then linked with other peer objects into arch/x86/boot/compressed/vmlinux. Meanwhile, a file called setup.bin is produced under arch/x86/boot. There may be an optional third file that has relocation info, depending on the configuration of CONFIG_X86_NEED_RELOCS.
A host program called build, provided by the kernel, builds these two (or three) parts into the final bzImage file.
Dependency tracking
Kbuild tracks three kinds of dependencies:
- All prerequisite files (both *.c and *.h)
- CONFIG_ options used in all prerequisite files
- Command-line dependencies used to compile the target
The first one is easy to understand, but what about the second and third? Kernel developers often see code pieces like this:
#ifdef CONFIG_SMP __boot_cpu_id = cpu; #endif
When CONFIG_SMP changes, this piece of code should be recompiled. The command line for compiling a source file also matters, because different command lines may result in different object files.
When a .c file uses a header file via a #include directive, you need write a rule like this:
main.o: defs.h recipe...
When managing a large project, you need a lot of these kinds of rules; writing them all would be tedious and boring. Fortunately, most modern C compilers can write these rules for you by looking at the #include lines in the source file. For the GNU Compiler Collection (GCC), it is just a matter of adding a command-line parameter: -MD depfile
# In scripts/Makefile.lib c_flags = -Wp,-MD,$(depfile) $(NOSTDINC_FLAGS) $(LINUXINCLUDE) \ -include $(srctree)/include/linux/compiler_types.h \ $(__c_flags) $(modkern_cflags) \ $(basename_flags) $(modname_flags)
This would generate a .d file with content like:
init_task.o: init/init_task.c include/linux/kconfig.h \ include/generated/autoconf.h include/linux/init_task.h \ include/linux/rcupdate.h include/linux/types.h \ ...
Then the host program fixdep takes care of the other two dependencies by taking the depfile and command line as input, then outputting a .<target>.cmd file in makefile syntax, which records the command line and all the prerequisites (including the configuration) for a target. It looks like this:
# The command line used to compile the target cmd_init/init_task.o := gcc -Wp,-MD,init/.init_task.o.d -nostdinc ... ... # The dependency files deps_init/init_task.o := \ $(wildcard include/config/posix/timers.h) \ $(wildcard include/config/arch/task/struct/on/stack.h) \ $(wildcard include/config/thread/info/in/task.h) \ ... include/uapi/linux/types.h \ arch/x86/include/uapi/asm/types.h \ include/uapi/asm-generic/types.h \ ...
A .<target>.cmd file will be included during recursive make, providing all the dependency info and helping to decide whether to rebuild a target or not.
The secret behind this is that fixdep will parse the depfile (.d file), then parse all the dependency files inside, search the text for all the CONFIG_ strings, convert them to the corresponding empty header file, and add them to the target's prerequisites. Every time the configuration changes, the corresponding empty header file will be updated, too, so kbuild can detect that change and rebuild the target that depends on it. Because the command line is also recorded, it is easy to compare the last and current compiling parameters.
Looking ahead
Kconfig/kbuild remained the same for a long time until the new maintainer, Masahiro Yamada, joined in early 2017, and now kbuild is under active development again. Don't be surprised if you soon see something different from what's in this article.