Ship precompiled gems for linux
Hello! If you're arriving here to give feedback on the v1.11.0 Release Candidates, please make sure to:
- tell us which linux distro you're using!
- (If you can) a docker image in which we can reproduce the failure you're seeing!
Thanks so much for using Nokogiri and for giving us feedback on this experiment!
See #1571 for the PR from @larskanis that enables building precompiled libraries for linux.
This issue is for discussion on how to ship those precompiled libraries in a gem for linux users in a way that won't break the installation experience for people.
Some things that we'll need to figure out were mentioned in #1571 by @larskanis:
Bundler isn't clever at selection of unsupported ruby versions. See Add two specs for choosing platform specific gems rubygems/bundler#6247 . This requires users to explicit opt-out of binary gems, when they want to run nokogiri on ruby-trunk.
There is a known issue on Alpine Linux: Cross building doesn't work for alpine3.7 rake-compiler/rake-compiler-dock#20 . I did a manual test on the alpine docker image and it reproducible fails at
Nokogiri::XML::Reader#namespacessomewhere within the tests: https://gist.github.com/larskanis/94920901e6466c9383360ee1346f9f34 . I didn't analyze the reason for the segfault so far, but I think we should before the release.
And he later mentioned:
Issue 1 in the above comment will probably finally be fixed by the next bundler release. See rubygems/bundler#7522 . This way bundler will select the platform=ruby gem automatically, if fat binary gem doesn't fit to the ruby release.
I think we'll probably also be able to fix / work around issue 2, so that I hope we'll be able to ship binary releases some day.
Finally, I'd like for VersionInfo to contain some indication that the user is using precompiled libraries. A patch that worked at one point on the #1571 branch (but now needs to be rewritten) is:
From 20f69908732b50258be0ee18e1d78b841d462dc4 Mon Sep 17 00:00:00 2001 From: Mike Dalessio <mike.dalessio@gmail.com> Date: Sun, 13 Jan 2019 01:15:06 -0500 Subject: [PATCH] `nokogiri -v` emits info about precompiled binaries It may be useful information to be able to tell if the libraries were compiled on installation, or whether they were precompiled in a fat binary native gem. --- Rakefile | 3 +++ ext/nokogiri/nokogiri.c | 6 ++++++ lib/nokogiri/version.rb | 6 ++++++ 3 files changed, 15 insertions(+) diff --git a/Rakefile b/Rakefile index 5ef937d..72edb7e 100644 --- a/Rakefile +++ b/Rakefile @@ -383,6 +383,9 @@ task :cross do raise "rake-compiler has not installed any cross rubies. Use rake-compiler-dock or 'rake gem:windows' for building binary windows gems." end + ENV['CFLAGS'] ||= "" + ENV['CFLAGS'] += " -DNOKOGIRI_USE_PRECOMPILED_NATIVE" + CROSS_RUBIES.each do |cross_ruby| task "tmp/#{cross_ruby.platform}/nokogiri/#{cross_ruby.ver}/nokogiri.so" do |t| # To reduce the gem file size strip mingw32 dlls before packaging diff --git a/ext/nokogiri/nokogiri.c b/ext/nokogiri/nokogiri.c index 740b30d..5b32500 100644 --- a/ext/nokogiri/nokogiri.c +++ b/ext/nokogiri/nokogiri.c @@ -104,6 +104,12 @@ void Init_nokogiri() rb_const_set(mNokogiri, rb_intern("NOKOGIRI_LIBXSLT_PATCHES"), Qnil); #endif +#ifdef NOKOGIRI_USE_PRECOMPILED_NATIVE + rb_const_set(mNokogiri, rb_intern("NOKOGIRI_USE_PRECOMPILED_NATIVE"), Qtrue); +#else + rb_const_set(mNokogiri, rb_intern("NOKOGIRI_USE_PRECOMPILED_NATIVE"), Qfalse); +#endif + #ifdef LIBXML_ICONV_ENABLED rb_const_set(mNokogiri, rb_intern("LIBXML_ICONV_ENABLED"), Qtrue); #else diff --git a/lib/nokogiri/version.rb b/lib/nokogiri/version.rb index 44c076a..c05e1a0 100644 --- a/lib/nokogiri/version.rb +++ b/lib/nokogiri/version.rb @@ -46,6 +46,10 @@ module Nokogiri NOKOGIRI_USE_PACKAGED_LIBRARIES end + def libxml2_using_precompiled_native? + NOKOGIRI_USE_PRECOMPILED_NATIVE + end + def warnings warnings = [] @@ -78,6 +82,7 @@ module Nokogiri vi["libxml"] = {}.tap do |libxml| if libxml2_using_packaged? libxml["source"] = "packaged" + libxml["precompiled"] = libxml2_using_precompiled_native? libxml["patches"] = NOKOGIRI_LIBXML2_PATCHES else libxml["source"] = "system" @@ -89,6 +94,7 @@ module Nokogiri vi["libxslt"] = {}.tap do |libxslt| if libxml2_using_packaged? libxslt["source"] = "packaged" + libxslt["precompiled"] = libxml2_using_precompiled_native? libxslt["patches"] = NOKOGIRI_LIBXSLT_PATCHES else libxslt["source"] = "system" -- 2.17.1
And tagging @tenderlove for visibility because he had some thoughts on the previous #1571 thread.