In yocto (poky) why is the layers config in the build/ folder?

Question

I'm new to yocto. I'm trying to learn how the packages are added, how to create new layers and so on... just poking around. Started by cloning poky and playing around.

To my understanding, the bblayers.conf file is critical to the project configuration and what you end up building (what layers and packages go into your final image).

This might be the wrong assumption, but I also have a feeling that the build/ folder is where things you build (bitbake) stay. Images, lots of things needed to build them, a big cache of stuff... You can delete it and rebuild it if you somehow broke it. Or you can just copy everything without the build/ folder and continue working on a different computer.

Apparently it's not quite the case. The build/conf/ folder has the important .conf files like the bblayers.conf.

Can someone explain why is this the case? Is there an elegant way to separate the project config and the build folder?

Acetaldehyde

See also #34301032 which is a closely related question. – Weald

Answer 1

There are a couple layers to the Yocto Project, mainly:

• BSPDIR: TOPDIR (build), sources, setup-environment
• BSPDIR/setup-environment: initial all the variable to for bitbake;
• BSPDIR/sources: meta-data/
• TOPDIR: conf/ sstate-cache/ cache/ tmp/ downloads/
• TOPDIR/downloads: recipe fetched packages;

 

• TOPDIR/conf/ : stored all the configuration. Mainly bblayers.conf, local.conf, sanity_info;
• TOPDIR/conf/bblayers.conf: stored all the path to meta-data that will be loaded;
• TOPDIR/conf/local.conf: configuration to build
• TOPDIR/conf/sanity_info: path double check to make sure that all the path used in the last compile match the current compile;
• TOPDIR/tmp/: Where all the compiling and building work happen

In BSPDIR/sources/poky/meta/conf/bitbake.conf

sources/poky/meta/conf/bitbake.conf:TMPDIR ?= "${TOPDIR}/tmp"
sources/poky/meta/conf/bitbake.conf:PERSISTENT_DIR = "${TOPDIR}/cache"
sources/poky/meta/conf/bitbake.conf:DL_DIR ?= "${TOPDIR}/downloads"
sources/poky/meta/conf/bitbake.conf:SSTATE_DIR ?= "${TOPDIR}/sstate-cache"

TOPDIR is where you initialize when run setup-environment or oe-init-build-env; All the other bitbake configuration environment variable can be changed based on your need in conf/local.conf;

e.g. modify conf/local.conf to change the downloads directory from TOPDIR/downloads;

DL_DIR ?= "/home/downloads/"

To create new layer, please watch this video: https://www.youtube.com/watch?v=3HsaoVqX7dg

Quinquennium

So it looks like the important thing in the TOPDIR is the conf directory (that I can't change) and I can configure the rest to be somewhere else? So it would be perfectly sensible to have a folder called build to contain tmp, downloads, cache, sstate-cache and a different one called project (or something else) that contains the conf folder? Will I then be able to just delete my build folder and regenerate it from project - and it gives me the advantage of clearly separating my config files from the tmp,cache,binaries etc? – Acetaldehyde

You are correct. Your project will be topdir and contains conf. Then in local.conf, set the path for TMPDIR, PERSISTENT_DIR, DL_DIR, and SSTATE_DIR to path/to/build/ . Most of the time we only delete the TMP folder instead of everything because the cache and download will make compiling faster for next time. – Quinquennium

It seems to work keeping conf in project, but I have to remove build/conf after each source oe-init-build-env. I've copied oe-init-build-env into my project root, and modified it so that it does not call os-setup-builddir, just creates directory BUILDDIR – Fingering

Answer 2

You might have followed the Yocto Project Quick Start Guide.

The earliest step in yocto after installing (cloning git repositories and installing packages) is to create your OE (OpenEmbedded) environment, which is done via:

source oe-init-build-env

This automatically creates and leads you to the build folder. Matter that you can give any directory of your system as parameter for this call (Reference Manual - Build Overview):

source oe-init-build-env [build_dir]

⤑ This is also the step, where your 'project config' is separated from the actual build folder.

⤑ As you assumed, in practice you would at most copy the layers and not the build folder. Even better is to leave sources from others in their git repositories and only copy and maintain your own layers.

Disquieting

I'm more interested in why is the config generated into the build dir? I would except the build/ folder (or any build_dir given) to be perishable (I can delete and regenerate without losing anything important), except for the conf folder within it. I don't get why is this the case (conf/ inside build/)? – Acetaldehyde

@VladV That's why you should local.conf as less as possible and rather set those settings in a custom machine.conf for example. Then you have to set only the machine variable in local.conf and the used layers in bblayers.conf – Disquieting

Answer 3

it is true an issue in the modern Yocto build system.

file bblayers.conf has to be synthesized based on MACHINE and DISTRO information using all provided (usually with the help of repo manifest file) layers by: collecting data from each available layer file layer.conf as well as conf/machine, conf/distro, images.

Instead bblayers.conf is usually copied over from the base layer conf/bblayers.conf location with the help of setup-environment script.

this approach provides no "one click" buildable environment but require maintainer/developer to look into readme to identify what layers are missing to be added to the build/conf/bblayers.conf.

Lucier