Multiplatform build support #140
Conversation
|
IMHO any |
|
It might be interesting to note here that I just pushed a github actions pipeline that builds and releases versioned images, so I no longer have to build releases locally on my (x86_64) development system. |
Hmm, valid point! The I adjusted the CI tasks, so that it looks green now. But it is only running the Is |
|
@larskanis I poked around a bit and it looks like we might be able to apply for free credits to run ARM hardware as an open-source project: https://github.com/WorksOnArm/GitHub-Runners Some related references: |
|
OK, I applied for ARM based Github Runners: WorksOnArm/GitHub-Runners#9 . Let's see what they think about it. For now |
|
@flavorjones What do you think about merging this? It allows at least to build and publish the images locally for |
|
@larskanis Let me just make sure I can still build everything, and if so I don't have any objections. I'll do that today, give me an hour or two. |
…NG.md And add a buildkit config which disables the garbage collection of the cache. Our images are too many and too fat, so that it gets deleted half way otherwise.
This way we can provide images for Intel/AMD based as well as ARM based systems. Image building is now prefixed by the host architecture "arm" or "x86" and the local host arch is loaded after build: ``` rake build:arm:arm-linux-musl # Build and load image for platform arm-linux-musl on linux/arm64 rake build:arm:arm64-darwin # Build and load image for platform arm64-darwin on linux/arm64 rake build:arm:jruby # Build and load image for JRuby on linux/arm64 ``` The foreign architecture is only built: ``` rake build:x86:arm-linux-musl # Build image for platform arm-linux-musl on linux/amd64 rake build:x86:arm64-darwin # Build image for platform arm64-darwin on linux/amd64 rake build:x86:jruby # Build image for JRuby on linux/amd64 ``` Only `docker buildx build` supports multiple platforms, so that it is used now, instead of the classic `docker build`. The number of images doubles with this patch, so that I get errors building all in parallel. Therefore the number of parallel tasks should be limited like: ``` rake -j10 build:all ```
Co-authored-by: Mike Dalessio <[email protected]>
|
@larskanis I rebased onto |
|
Mike, do you have an arm based system locally to create a docker buildx cluster as described in the contributing doc of this PR? I've got a new quite powerful laptop due to fundraiser on https://rubyinstaller.org . To my understanding, per platform images can not be simply appended after pushing. Either the two host platforms must be pushed by one buildx command or two images must be pushed separately with different names and then a manifest with links to them must be pushed. The rake task |
|
@larskanis Unfortunately, I do not own ARM hardware. Until we figure out how to deploy, we need to maintain the ability to ship the x86 platform images, which is why I rebased and updated the CI pipeline. |
|
I'm OK merging this. Let's continue the conversation about how to deploy these images in #78? |
Add support for multiple host platforms.
This way we can provide images for Intel/AMD based as well as ARM based systems.
Image building is now prefixed by the host architecture "arm" or "x86" and the local host arch is loaded after build:
The foreign architecture is only built:
Only
docker buildx buildsupports multiple platforms, so that it is used now, instead of the classicdocker build.The number of images doubles with this patch, so that I get errors building all in parallel.
Therefore the number of parallel tasks should be limited like: