[gold patch 8/8] Add framework for incremental updates.

Cary Coutant ccoutant@google.com
Tue Sep 28 20:00:00 GMT 2010
Previous message (by thread): [gold patch 7/8] Restructure Incremental_binary.
Next message (by thread): is_elf_format and friends
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
This patch adds the framework for doing the incremental update itself.
It's a big patch, but a lot of it is boilerplate from adding new classes
derived from Relobj and Dynobj to represent unchanged objects in an
incremental link.

I added a serial number to each input argument, and record the serial
number in the incremental info for each input file, in order to apply the
--incremental-changed/unchanged disposition reliably.

I also added a bit flag to the global symbol lists for input objects
indicating whether the symbol was defined in that object or not.

For now, I have disabled the .eh_frame_hdr generation and optimization
of the .eh_frame sections. In order to restore that, I may need to
record extra relocations for those sections.

I'm still working on a few more things before this is ready for use:
- Insertion of padding.
- Getting the right soname for shared libraries.
- Handling local symbols with dynamic relocations.
- Copying and applying relocations from unchanged inputs.
- String merge sections.
- COMDAT group handling.
- Symbol versions.
- Fill holes in .debug_info and .eh_frame sections.

-cary


	* archive.cc (Archive::include_member): Adjust call to report_object.
	(Add_archive_symbols::run): Track serial number of input argument.
	(Lib_group::include_member): Likewise.
	(Add_lib_group_symbols::run): Adjust call to report_object.
	* archive.h (Archive_member::arg_serial_): New data member.
	* dynobj.cc (Dynobj::Dynobj): Allow for NULL input_file.
	(Sized_dynobj::do_add_symbols): Track symbols for incremental links.
	(Sized_dynobj::do_for_all_local_got_entries): New function.
	* dynobj.h (Sized_dynobj::do_for_all_local_got_entries): New function.
	* fileread.cc (get_mtime): New function.
	* fileread.h (get_mtime): New function declaration.
	* gold.cc (queue_initial_tasks): Setup for incremental linking.
	(process_incremental_input): New function.
	(queue_middle_tasks): Add check for incremental update.
	* incremental-dump.cc (dump_incremental_inputs): Print serial numbers
	of input arguments.  Print more info for symbols.
	* incremental.cc (Output_section_incremental_inputs::update_data_size):
	New function.
	(Sized_incremental_binary::setup_readers): Setup readers for input
	files and maps of files added from libraries and scripts.
	(Sized_incremental_binary::check_input_args): New function.
	(Sized_incremental_binary::do_check_inputs): Check for changed script
	files.
	(Sized_incremental_binary::do_get_input_reader): Remove.
	(Sized_incremental_binary::do_file_has_changed): New function.
	(Sized_incremental_binary::do_init_layout): New function.
	(Sized_incremental_binary::do_reserve_layout): New function.
	(Sized_incremental_binary::get_symtab_view): New function.
	(Incremental_checker::can_incrementally_link_output_file): Remove.
	(Incremental_inputs::report_archive_begin): Add serial number of
	input argument.
	(Incremental_inputs::report_object): Likewise.
	(Incremental_inputs::report_script): Likewise.
	(Output_section_incremental_inputs::set_final_data_size): Count
	symbols from shared libraries.
	(Output_section_incremental_inputs::write_input_files): Write serial
	numbers for input file arguments.
	(Output_section_incremental_inputs::write_info_blocks): Map section
	index to original input section index.  Record def/undef flag for
	each global symbol.  Omit global symbols from shared libraries that
	are not used in the output symbol table.
	(Local_got_offset_visitor): Derive from new base class.  Change from
	functor to a visit callback.
	(Global_got_offset_visitor): Likewise.
	(Output_section_incremental_inputs::write_got_plt): Call new
	local GOT entry visitor instead of iterating over symbols.
	(Sized_incr_relobj::do_read_symbols): New function.
	(Sized_incr_relobj::do_layout): New function.
	(Sized_incr_relobj::do_layout_deferred_sections): New function.
	(Sized_incr_relobj::do_add_symbols): New function.
	(Sized_incr_relobj::do_should_include_member): New function.
	(Sized_incr_relobj::do_for_all_global_symbols): New function.
	(Sized_incr_relobj::do_for_all_local_got_entries): New function.
	(Sized_incr_relobj::do_section_size): New function.
	(Sized_incr_relobj::do_section_name): New function.
	(Sized_incr_relobj::do_section_contents): New function.
	(Sized_incr_relobj::do_section_flags): New function.
	(Sized_incr_relobj::do_section_entsize): New function.
	(Sized_incr_relobj::do_section_address): New function.
	(Sized_incr_relobj::do_section_type): New function.
	(Sized_incr_relobj::do_section_link): New function.
	(Sized_incr_relobj::do_section_info): New function.
	(Sized_incr_relobj::do_section_addralign): New function.
	(Sized_incr_relobj::do_initialize_xindex): New function.
	(Sized_incr_relobj::do_get_global_symbol_counts): New function.
	(Sized_incr_relobj::do_read_relocs): New function.
	(Sized_incr_relobj::do_gc_process_relocs): New function.
	(Sized_incr_relobj::do_scan_relocs): New function.
	(Sized_incr_relobj::do_count_local_symbols): New function.
	(Sized_incr_relobj::do_finalize_local_symbols): New function.
	(Sized_incr_relobj::do_set_local_dynsym_indexes): New function.
	(Sized_incr_relobj::do_relocate): New function.
	(Sized_incr_relobj::do_set_section_offset): New function.
	(Sized_incr_dynobj::do_read_symbols): New function.
	(Sized_incr_dynobj::do_layout): New function.
	(Sized_incr_dynobj::do_add_symbols): New function.
	(Sized_incr_dynobj::do_should_include_member): New function.
	(Sized_incr_dynobj::do_for_all_global_symbols): New function.
	(Sized_incr_dynobj::do_for_all_local_got_entries): New function.
	(Sized_incr_dynobj::do_section_size): New function.
	(Sized_incr_dynobj::do_section_name): New function.
	(Sized_incr_dynobj::do_section_contents): New function.
	(Sized_incr_dynobj::do_section_flags): New function.
	(Sized_incr_dynobj::do_section_entsize): New function.
	(Sized_incr_dynobj::do_section_address): New function.
	(Sized_incr_dynobj::do_section_type): New function.
	(Sized_incr_dynobj::do_section_link): New function.
	(Sized_incr_dynobj::do_section_info): New function.
	(Sized_incr_dynobj::do_section_addralign): New function.
	(Sized_incr_dynobj::do_initialize_xindex): New function.
	(Sized_incr_dynobj::do_get_global_symbol_counts): New function.
	(make_sized_incremental_object): New function.
	(Incremental_library::copy_unused_symbols): New function.
	(Incremental_library::do_for_all_unused_symbols): New function.
	* incremental.h (Incremental_checker): Remove class.
	(Incremental_input_entry::arg_serial): New function.
	(Incremental_input_entry::arg_serial_): New data member.  Adjust ctor.
	(Script_info): Move class here from script.h.
	(Script_info::filename): New function.
	(Script_info::filename_): New data member.
	(Incremental_script_entry): Add arg_serial parameter to ctor; adjust
	all callers.
	(Incremental_object_entry): Likewise.
	(Incremental_object_entry::add_input_section): Push new input section
	onto list.
	(Incremental_object_entry::get_input_section_index): New function.
	(Incremental_object_entry::Input_section): Add constructor; rename
	data members; adjust callers.
	(Incremental_archive_entry): Add arg_serial parameter to ctor; adjust
	all callers.
	(Incremental_inputs::report_archive_begin): Add arg_serial parameter.
	(Incremental_inputs::report_object): Likewise.
	(Incremental_inputs::report_script): Likewise.
	(Incremental_inputs_reader::arg_serial): New function.
	(Incremental_inputs_reader::Global_symbol_info::is_def): New data
	member.
	(Incremental_inputs_reader::get_global_symbol_info): Add shared library
	case.
	(Incremental_inputs_reader::input_file_offset): New function.
	(Incremental_inputs_reader::input_file_index): New function.
	(Incremental_inputs_reader::input_file): Rewrite.
	(Incremental_inputs_reader::input_file_at_offset): New function.
	(Incremental_binary::check_inputs): Add cmdline parameter.
	(Incremental_binary::Input_reader::arg_serial): New function.
	(Incremental_binary::Input_reader::get_unused_symbol_count): New
	function.
	(Incremental_binary::Input_reader::get_unused_symbol): New function.
	(Incremental_binary::Input_reader::do_arg_serial): New function.
	(Incremental_binary::Input_reader::do_get_unused_symbol_count): New
	function.
	(Incremental_binary::Input_reader::do_get_unused_symbol): New function.
	(Incremental_binary::get_input_reader): Remove.
	(Incremental_binary::input_file_count): New function.
	(Incremental_binary::get_input_reader): New function.
	(Incremental_binary::file_has_changed): New function.
	(Incremental_binary::get_input_argument): New function.
	(Incremental_binary::get_library): New function.
	(Incremental_binary::get_script_info): New function.
	(Incremental_binary::init_layout): New function.
	(Incremental_binary::reserve_layout): New function.
	(Incremental_binary::output_file): New function.
	(Incremental_binary::do_check_inputs): Add cmdline parameter.
	(Incremental_binary::do_file_has_changed): New function.
	(Incremental_binary::do_init_layout): New function.
	(Incremental_binary::do_reserve_layout): New function.
	(Incremental_binary::do_input_file_count): New function.
	(Incremental_binary::do_get_input_reader): Change signature.
	(Incremental_binary::input_args_map_): New data member.
	(Incremental_binary::library_map_): New data member.
	(Incremental_binary::script_map_): New data member.
	(Sized_incremental_binary::output_section): New function.
	(Sized_incremental_binary::inputs_reader): Add const.
	(Sized_incremental_binary::symtab_reader): Add const.
	(Sized_incremental_binary::relocs_reader): Add const.
	(Sized_incremental_binary::got_plt_reader): Add const.
	(Sized_incremental_binary::get_symtab_view): New function.
	(Sized_incremental_binary::Inputs_reader): New typedef.
	(Sized_incremental_binary::Input_entry_reader): New typedef.
	(Sized_incremental_binary::do_check_inputs): Add cmdline parameter.
	(Sized_incremental_binary::do_file_has_changed): New function.
	(Sized_incremental_binary::do_init_layout): New function
	(Sized_incremental_binary::do_reserve_layout): New function
	(Sized_incremental_binary::Sized_input_reader): Remove typedefs.
	(Sized_incremental_binary::Sized_input_reader::do_arg_serial): New
	function.
	(Sized_incremental_binary::Sized_input_reader::
	do_get_unused_symbol_count): New function.
	(Sized_incremental_binary::Sized_input_reader::do_get_unused_symbol):
	New function.
	(Sized_incremental_binary::do_get_input_reader): Change signature.
	(Sized_incremental_binary::do_input_file_count): New function.
	(Sized_incremental_binary::section_map_): New data member.
	(Sized_incremental_binary::input_entry_readers_): New data member.
	(Sized_incr_relobj): New class.
	(Sized_incr_dynobj): New class.
	(make_sized_incremental_object): New function.
	(Incremental_library): New class.
	* layout.cc (Free_list::init): New function.
	(Free_list::remove): New function.
	(Free_list::allocate): New function.
	(Free_list::dump): New function.
	(Layout_task_runner::run): Check for existing output file when doing
	incremental update.
	(Layout::Layout): Initialize new data members.
	(Layout::set_incremental_base): New function.
	(Layout::init_fixed_output_section): New function.
	(Layout::layout_eh_frame): Disable .eh_frame_hdr for incremental links.
	(Layout::create_gold_note): Skip for incremental update.
	(Layout::set_section_offsets): Allocate from free space for incremental
	updates.
	(Layout::create_symtab_sections): Likewise.
	(Layout::create_shdrs): Likewise.
	(Layout::finish_dynamic_section): Don't check --as-needed during
	incremental updates.
	(Layout::init_fixed_output_section): Instantiate templates.
	* layout.h (Free_list): New class.
	(Layout::set_incremental_base): New function.
	(Layout::incremental_base): New function.
	(Layout::init_fixed_output_section): New function.
	(Layout::allocate): New function.
	(Layout::incremental_base_): New data member.
	(Layout::free_list_): New data member.
	* object.cc (Sized_relobj::do_layout): Don't report non-data input
	sections.
	(Sized_relobj::do_for_all_local_got_entries): New function.
	(Sized_relobj::write_local_symbols): Add symbol table file offset to
	relative local symbol offsets.
	* object.h (Got_offset_list::for_all_got_offsets): Change to call
	visitor class.
	(Object::Object): Handle case where input_file is NULL.
	(Object::~Object): Likewise.
	(Object::input_file): Likewise.
	(Object::lock): Likewise.
	(Object::unlock): Likewise.
	(Object::is_locked): Likewise.
	(Object::token): Likewise.
	(Object::release): Likewise.
	(Object::is_incremental): New function.
	(Object::get_mtime): New function.
	(Object::for_all_local_got_entries): New function.
	(Object::clear_view_cache_marks): Handle case where input_file is NULL.
	(Object::do_is_incremental): New function.
	(Object::do_get_mtime): New function.
	(Object::do_for_all_local_got_entries): New function.
	(Sized_relobj::local_got_offset_list): Remove.
	(Sized_relobj::do_for_all_local_got_entries): New function.
	(Sized_relobj::write_local_symbols): Change signature.
	(Sized_relobj::local_symbol_offset_): Now relative to symbol table.
	* options.cc (Input_argument::add_file): Set serial number of input
	argument.
	* options.h (Input_file_argument::set_arg_serial): New function.
	(Input_file_argument::arg_serial): New function.
	(Input_file_argument::arg_serial_): New data member; adjust ctors.
	(Input_arguments::add_file): Remove const.
	(Input_arguments::file_count_): New data member.
	* output.cc (Output_segment_headers::Output_segment_headers): Keep
	current size up to date.
	(Output_section::Input_section::current_data_size): New function.
	(Output_section::Output_section): Initialize new data members.
	(Output_section::add_input_section): Allocate from free space for
	incremental updates.
	(Output_section::add_output_section_data): Likewise.
	(Output_section::update_data_size): New function.
	(Output_section::set_fixed_layout): New function.
	(Output_section::reserve): New function.
	(Output_segment::set_section_addresses): Remove const.
	(Output_segment::set_section_list_addresses): Remove const.  Allocate
	from free space for incremental updates.
	* output.h (Output_data::current_data_size): Moved here from derived
	classes.
	(Output_data::pre_finalize_data_size): New function.
	(Output_data::update_data_size): New function.
	(Output_section_headers::update_data_size): New function.
	(Output_section_data_build::current_data_size): Move to base class.
	(Output_data_strtab::update_data_size): New function.
	(Output_section::Input_section::current_data_size): New function.
	(Output_section::current_data_size): Move to base class.
	(Output_section::set_fixed_layout): New function.
	(Output_section::has_fixed_layout): New function.
	(Output_section::reserve): New function.
	(Output_section::update_data_size): New function.
	(Output_section::has_fixed_layout_): New data member.
	(Output_section::free_list_): New data member.
	(Output_segment::set_section_addresses): Remove const.
	(Output_segment::set_section_list_addresses): Remove const.
	* plugin.cc (Sized_pluginobj::do_for_all_local_got_entries): New
	function.
	* plugin.h (Sized_pluginobj::do_for_all_local_got_entries): New
	function.
	* readsyms.cc (Read_symbols::do_read_symbols): Track serial number of
	input argument.
	(Add_symbols::run): Likewise.
	(Read_member::~Read_member): New function.
	(Read_member::is_runnable): New function.
	(Read_member::locks): New function.
	(Read_member::run): New function.
	(Add_base_symbols::~Add_base_symbols): New function.
	(Add_base_symbols::is_runnable): New function.
	(Add_base_symbols::locks): New function.
	(Add_base_symbols::run): New function.
	(Check_script::~Check_script): New function.
	(Check_script::is_runnable): New function.
	(Check_script::locks): New function.
	(Check_script::run): New function.
	(Check_library::~Check_library): New function.
	(Check_library::is_runnable): New function.
	(Check_library::locks): New function.
	(Check_library::run): New function.
	* readsyms.h (Read_member): New class.
	(Add_base_symbols): New class.
	(Check_script): New class.
	(Check_library): New class.
	* reloc.cc (Read_relocs::locks): Handle incremental objects that don't
	have input files.
	(Scan_relocs::locks): Likewise.
	(Relocate_task::locks): Likewise.
	(Sized_relobj::do_relocate): Pass file offset of symtab section to
	write_local_symbols.
	* script.cc (read_input_script): Track serial number of input
	argument.
	* script.h (Script_info): Move to incremental.h.
	* symtab.cc (Symbol_table::add_from_incrobj): New function.
	Instantiate templates.
	* symtab.h (Symbol_table::add_from_incrobj): New function.
	(Symbol_table::set_file_offset): New function.
---
 gold/archive.cc          |   15 +-
 gold/archive.h           |    4 +-
 gold/dynobj.cc           |   40 ++-
 gold/dynobj.h            |    5 +
 gold/fileread.cc         |   19 +
 gold/fileread.h          |    6 +
 gold/gold.cc             |  250 +++++++++-
 gold/incremental-dump.cc |   94 +++--
 gold/incremental.cc      | 1158 +++++++++++++++++++++++++++++++++++++++++++---
 gold/incremental.h       |  800 ++++++++++++++++++++++++++++----
 gold/layout.cc           |  332 ++++++++++++--
 gold/layout.h            |   64 +++
 gold/object.cc           |   38 ++-
 gold/object.h            |  314 ++++++++-----
 gold/options.cc          |    3 +-
 gold/options.h           |   24 +-
 gold/output.cc           |  204 +++++++-
 gold/output.h            |   83 +++-
 gold/plugin.cc           |   10 +
 gold/plugin.h            |    5 +
 gold/readsyms.cc         |  185 ++++++++-
 gold/readsyms.h          |  200 ++++++++
 gold/reloc.cc            |   15 +-
 gold/script.cc           |    6 +-
 gold/script.h            |   25 -
 gold/symtab.cc           |   72 +++
 gold/symtab.h            |   14 +
 27 files changed, 3517 insertions(+), 468 deletions(-)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 8
Type: text/x-patch
Size: 174552 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/binutils/attachments/20100928/d5efa5a0/attachment.bin>
Previous message (by thread): [gold patch 7/8] Restructure Incremental_binary.
Next message (by thread): is_elf_format and friends
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the Binutils mailing list