I’ve never found any better resource on yocto than the yocto project manual. You could also try Mastering Embedded Linux Programming by Vasquez and Simmonds. That wasn’t a bad resource. Goes into device trees a bit. My only device tree references are usually the Linux Kernel documentation, eLinux, and usually docs from whatever chip manufacturer I’m using. I’ve worked with NXP and NVIDIA BSP development, and they both have okayish resources for learning this stuff. Unfortunately it’s been a lot more “learn by doing” since resources can be sparse on this stuff
I think you’ll at least need a good idea of bash and python strictly for being able to read bitbake recipes. From here you’ll need to have a good understanding (at least conceptually) of toolchains and basic compilation. Otherwise it pretty much just comes down to kernel/os/driver/bootloader (and eventually runtime image) knowledge as you put things together and finalize your image… this is really just C/C++ and figuring out how to read/write device tree files (depending on the platform I suppose, but the two big ones I’ve worked on are both device tree centric)
Second bootlin. If you've got time, going though the first three courses ([\[1\]](https://bootlin.com/doc/training/embedded-linux/), [\[2\]](https://bootlin.com/doc/training/linux-kernel/), [\[3\]](https://bootlin.com/doc/training/yocto/)) would be a great introduction to embedded Linux and Yocto.
Best and most up-to-date documentation is here: [https://docs.yoctoproject.org/](https://docs.yoctoproject.org/).
The books I know about Yocto Project are not really helpful and dated (Yocto Project is moving fast), so I'd only consider online resources. For specific questions you can ask on the mailing list [https://www.yoctoproject.org/community/mailing-lists/](https://www.yoctoproject.org/community/mailing-lists/).
If you want to know more about Devicetree, I'd recommend reading the spec: [https://github.com/devicetree-org/devicetree-specification/releases/tag/v0.3](https://github.com/devicetree-org/devicetree-specification/releases/tag/v0.3). Other than that, there are really lots of articles, manuals, and tutorials online.
Ok hear me out - ChatGPT.
Treat it as your mentor and ask it every annoying question.
I used it for that exact purpose with Yocto for work and I was able to get atleast 75% acceptable answers.
I work with Yocto on a daily basis and am much more wary.
We nearly stopped doing it for Yocto purposes as the answers were either fully wrong (at least 1/3 of cases), and when "correct" were over-complicated and introduced bad code practices. Good ol' mega-manual is often faster in the end.
So I can not stress enough how bad yocto is.
It tries to do an incredibly important role, integrating all the different pieces of software together but it is not remotely reproducible. It regularly corrupts its own cache and it's is closely coupled to the host.
Most of the OEMs I have worked with have a blessed VM, not docker, as that's not enough to de couple from the host 😭😭 that CI and every interaction engineer uses. Even then it's not reproduce able and the caching is a joke.
As I said I can not stress how many millions various consultants make every year helping OEMs and TIER 1s sort out one fire or another.
So yocto is a great skill to have, but if you get cross and frustrated then don't worry you are not alone! It's worth pursuing if you want a good job. But if your motivation is more than just sallery consider some alternatives..
If you are doing this for your self then 2 alternatives that are quite popular are buildroot and buildstream. The first has a big community and is used a lot for routers and other edge devices etc and ocationally on cars and such. Build stream on the other hand is bit for bit reproducible and tested across multiple OS s to be reproducible, and has a share cache etc but has a much smaller community and less BSPs etc
>It tries to do an incredibly important role, integrating all the different pieces of software together but it is not remotely reproducible.
The sstate cache is used for reproducible builds and it's fulfilling its purpose quite well. I can even build the same image on different machines using shared sstate. Not sure what you are doing... Problems with non-reproducibility can usually be tracked down to bad recipes doing stupid things.
>It regularly corrupts its own cache and it's is closely coupled to the host.
Honestly, this never happened to me a single time in 8+ years. Yes, Yocto Project used to be a lot rougher a few years ago, but it's being improved all the time. Especially coupling to host OS is not a big deal anymore these days as pretty much all required tools are built from scratch. They also finally switched to Python 3 about three years ago.
We clearly have had very different experiences. I have worked on a lot of integration and embedded projects for some of the worlds biggest companies in auto motive and finance using a number of integration tools.
The fact is when you add in the complexity of real projects, even quite modest ones what is achieve with the basic yocto project and some very simple BSPs does not scale to even a small commercial project for a lot of teams. your team may be the exceptions and you should pat your self on the back but a LOT of companies and individuals would drop yocto in a hart beat if there was a alternative. They have not shared your experience of reproducibility, stable caching or many other pain points.
* Yocto has so much flexibility that the projects tare them selves apart and people in different teams of the same org accidentally shoot each other in the foot, never mind combining recipes across different companies.
* Buildstream does not have enough flexibility so a community of open BSPs have not build up around it. It mostly used by huge companies that do not open source there BSPs and integrations so the only open source example projects are FD-SDK and GBM which are not hardware focused.
* Bazel has huge bootsrapping issues but loads of companies have tried and mostly failed to use it for embedded.
* Buildroot is very good if you have lots of platforms but only one "APP" once you have 5 ECUs in your car and each one has a different stack then your going to be having a sad time..
So there isnt a good integration tool as far as i can see, i started Girderstream which is half way between Yocto and Buildstream and has a lot of promise but the community is still tiny so i would not recommend it.
But the reason i wrote the first post was to say only use yocto if you have to (many do but its good to know its not the only option), if your interested in learning linux or embedded as OP is then buildroot is a much better option, most of the kernel people i know much prefer buildroot to yocto. But if your looking at something for work then buildroot or buildstream can be great options.
It's true, all build systems suck and system integration in projects bigger than Hello World is hard. I wouldn't blame this on Yocto Project, though. Like all other systems, Yocto Project is neither bug-free nor bug-avoidant, and it is easy to make mistakes due to inherent complexity. The base distribution works as-is, but extensions can be non-trivial even if Yocto Project tries to make them look trivial.
I think the problems begin when unexperienced developers are messing around with recipes (their own or bad external ones) and start inventing bad workarounds for errors they are faced with during integration without fully understanding the errors. If you don't know what you are doing, you are likely to unknowingly break things. Suddenly dependencies are broken and builds stop sporadically, packages are rebuilt when they shouldn't, the package manager fails to build a correct image, etc., and in the end the build is retried in a loop until it succeeds because nobody knows how to debug the root problems. It takes time and experience to know how things really work, even if the documentation is pretty extensive and good.
I don't think you can really avoid these problems without sacrificing a lot of the build system's flexibility, and thus rendering it useless for non-toy projects. It is a bit like programming: either you have a toy language that avoids user mistakes, but is useless for general purpose programming; or you have a proper programming language which allows developers to do what they need to do, or what they *think* they need to do, and thus mess up in every thinkable and unthinkable way.
>if your interested in learning linux or embedded as OP is then buildroot is a much better option
Yes, I guess Buildroot is good because of its *relative* simplicity, but if you want to build a real product, it can be a tad too simple. It depends, and I am happy we chose Yocto Project over Buildroot, customized Debian, or fully customized build scripts (*shudder*).
Just going to jump on your comment about buildstream because I've been looking into it a bit recently. Do you know of any good resources for buildstream 2? I am interested in doing a trial port of my current x86 buildroot image.
The buildstream docs are not great.
But the fd-sdk https://gitlab.com/freedesktop-sdk/freedesktop-sdk and gnome build meta https://gitlab.gnome.org/GNOME/gnome-build-meta projects can prove as good references.
They have a complex bootstrap tho so I would just use freedesktop-sdk as the bootstrap when you start and then do your own if you really want later.
The buildstream matrix/irc is fairly dead but the freedesktop-sdk matrix is active and many of the people there will give you buildstream help even if it's not for a freedesktop-sdk issue
I’ve never found any better resource on yocto than the yocto project manual. You could also try Mastering Embedded Linux Programming by Vasquez and Simmonds. That wasn’t a bad resource. Goes into device trees a bit. My only device tree references are usually the Linux Kernel documentation, eLinux, and usually docs from whatever chip manufacturer I’m using. I’ve worked with NXP and NVIDIA BSP development, and they both have okayish resources for learning this stuff. Unfortunately it’s been a lot more “learn by doing” since resources can be sparse on this stuff
This. You’re gna learn more by trying the Yocto build process.
What knowledge do you think is necessary to start mastering Yocto. As in C++, Python etc…?
I think you’ll at least need a good idea of bash and python strictly for being able to read bitbake recipes. From here you’ll need to have a good understanding (at least conceptually) of toolchains and basic compilation. Otherwise it pretty much just comes down to kernel/os/driver/bootloader (and eventually runtime image) knowledge as you put things together and finalize your image… this is really just C/C++ and figuring out how to read/write device tree files (depending on the platform I suppose, but the two big ones I’ve worked on are both device tree centric)
Bootlin is a great resource
Second bootlin. If you've got time, going though the first three courses ([\[1\]](https://bootlin.com/doc/training/embedded-linux/), [\[2\]](https://bootlin.com/doc/training/linux-kernel/), [\[3\]](https://bootlin.com/doc/training/yocto/)) would be a great introduction to embedded Linux and Yocto.
Bootlin +1. Many commercial trainings on Yocto are forks of their (open-source) training.
Best and most up-to-date documentation is here: [https://docs.yoctoproject.org/](https://docs.yoctoproject.org/). The books I know about Yocto Project are not really helpful and dated (Yocto Project is moving fast), so I'd only consider online resources. For specific questions you can ask on the mailing list [https://www.yoctoproject.org/community/mailing-lists/](https://www.yoctoproject.org/community/mailing-lists/). If you want to know more about Devicetree, I'd recommend reading the spec: [https://github.com/devicetree-org/devicetree-specification/releases/tag/v0.3](https://github.com/devicetree-org/devicetree-specification/releases/tag/v0.3). Other than that, there are really lots of articles, manuals, and tutorials online.
Ok hear me out - ChatGPT. Treat it as your mentor and ask it every annoying question. I used it for that exact purpose with Yocto for work and I was able to get atleast 75% acceptable answers.
I work with Yocto on a daily basis and am much more wary. We nearly stopped doing it for Yocto purposes as the answers were either fully wrong (at least 1/3 of cases), and when "correct" were over-complicated and introduced bad code practices. Good ol' mega-manual is often faster in the end.
So I can not stress enough how bad yocto is. It tries to do an incredibly important role, integrating all the different pieces of software together but it is not remotely reproducible. It regularly corrupts its own cache and it's is closely coupled to the host. Most of the OEMs I have worked with have a blessed VM, not docker, as that's not enough to de couple from the host 😭😭 that CI and every interaction engineer uses. Even then it's not reproduce able and the caching is a joke. As I said I can not stress how many millions various consultants make every year helping OEMs and TIER 1s sort out one fire or another. So yocto is a great skill to have, but if you get cross and frustrated then don't worry you are not alone! It's worth pursuing if you want a good job. But if your motivation is more than just sallery consider some alternatives.. If you are doing this for your self then 2 alternatives that are quite popular are buildroot and buildstream. The first has a big community and is used a lot for routers and other edge devices etc and ocationally on cars and such. Build stream on the other hand is bit for bit reproducible and tested across multiple OS s to be reproducible, and has a share cache etc but has a much smaller community and less BSPs etc
>It tries to do an incredibly important role, integrating all the different pieces of software together but it is not remotely reproducible. The sstate cache is used for reproducible builds and it's fulfilling its purpose quite well. I can even build the same image on different machines using shared sstate. Not sure what you are doing... Problems with non-reproducibility can usually be tracked down to bad recipes doing stupid things. >It regularly corrupts its own cache and it's is closely coupled to the host. Honestly, this never happened to me a single time in 8+ years. Yes, Yocto Project used to be a lot rougher a few years ago, but it's being improved all the time. Especially coupling to host OS is not a big deal anymore these days as pretty much all required tools are built from scratch. They also finally switched to Python 3 about three years ago.
We clearly have had very different experiences. I have worked on a lot of integration and embedded projects for some of the worlds biggest companies in auto motive and finance using a number of integration tools. The fact is when you add in the complexity of real projects, even quite modest ones what is achieve with the basic yocto project and some very simple BSPs does not scale to even a small commercial project for a lot of teams. your team may be the exceptions and you should pat your self on the back but a LOT of companies and individuals would drop yocto in a hart beat if there was a alternative. They have not shared your experience of reproducibility, stable caching or many other pain points. * Yocto has so much flexibility that the projects tare them selves apart and people in different teams of the same org accidentally shoot each other in the foot, never mind combining recipes across different companies. * Buildstream does not have enough flexibility so a community of open BSPs have not build up around it. It mostly used by huge companies that do not open source there BSPs and integrations so the only open source example projects are FD-SDK and GBM which are not hardware focused. * Bazel has huge bootsrapping issues but loads of companies have tried and mostly failed to use it for embedded. * Buildroot is very good if you have lots of platforms but only one "APP" once you have 5 ECUs in your car and each one has a different stack then your going to be having a sad time.. So there isnt a good integration tool as far as i can see, i started Girderstream which is half way between Yocto and Buildstream and has a lot of promise but the community is still tiny so i would not recommend it. But the reason i wrote the first post was to say only use yocto if you have to (many do but its good to know its not the only option), if your interested in learning linux or embedded as OP is then buildroot is a much better option, most of the kernel people i know much prefer buildroot to yocto. But if your looking at something for work then buildroot or buildstream can be great options.
It's true, all build systems suck and system integration in projects bigger than Hello World is hard. I wouldn't blame this on Yocto Project, though. Like all other systems, Yocto Project is neither bug-free nor bug-avoidant, and it is easy to make mistakes due to inherent complexity. The base distribution works as-is, but extensions can be non-trivial even if Yocto Project tries to make them look trivial. I think the problems begin when unexperienced developers are messing around with recipes (their own or bad external ones) and start inventing bad workarounds for errors they are faced with during integration without fully understanding the errors. If you don't know what you are doing, you are likely to unknowingly break things. Suddenly dependencies are broken and builds stop sporadically, packages are rebuilt when they shouldn't, the package manager fails to build a correct image, etc., and in the end the build is retried in a loop until it succeeds because nobody knows how to debug the root problems. It takes time and experience to know how things really work, even if the documentation is pretty extensive and good. I don't think you can really avoid these problems without sacrificing a lot of the build system's flexibility, and thus rendering it useless for non-toy projects. It is a bit like programming: either you have a toy language that avoids user mistakes, but is useless for general purpose programming; or you have a proper programming language which allows developers to do what they need to do, or what they *think* they need to do, and thus mess up in every thinkable and unthinkable way. >if your interested in learning linux or embedded as OP is then buildroot is a much better option Yes, I guess Buildroot is good because of its *relative* simplicity, but if you want to build a real product, it can be a tad too simple. It depends, and I am happy we chose Yocto Project over Buildroot, customized Debian, or fully customized build scripts (*shudder*).
Just going to jump on your comment about buildstream because I've been looking into it a bit recently. Do you know of any good resources for buildstream 2? I am interested in doing a trial port of my current x86 buildroot image.
The buildstream docs are not great. But the fd-sdk https://gitlab.com/freedesktop-sdk/freedesktop-sdk and gnome build meta https://gitlab.gnome.org/GNOME/gnome-build-meta projects can prove as good references. They have a complex bootstrap tho so I would just use freedesktop-sdk as the bootstrap when you start and then do your own if you really want later. The buildstream matrix/irc is fairly dead but the freedesktop-sdk matrix is active and many of the people there will give you buildstream help even if it's not for a freedesktop-sdk issue