SDK refactoring - drop some of the Boost library builds

Programmers discuss here anything related to FreeOrion programming. Primarily for the developers to discuss.

Moderator: Committer

Post Reply
Message
Author
User avatar
mem359
Dyson Forest
Posts: 214
Joined: Sun Jun 08, 2014 1:18 am

SDK refactoring - drop some of the Boost library builds

#1 Post by mem359 » Mon Jun 05, 2017 1:20 pm

For the SDK builds, we have a list of Boost library builds.
I am proposing making that list smaller.

At least for the Mac, these libraries are not being used by FO, or by any of the other Boost libraries that are being used.

I wanted to get feedback, in case there are dependencies in the Windows build, or if there are plans to use these libraries in the future.

The advantage would be less compiling (30 min instead of 40 on my computer for the SDK), and smaller code size (fewer includes and libraries being copied).

I would take care of submitting a PR, which would only have a change in CMakeLists.txt.
(No Xcode or MSVC changes.)

------

Current Boost libraries being built, and want to keep in the SDK:
chrono, date_time, filesystem, iostreams, locale, log, log_setup, python, regex, serialization, signals, system, thread

Already being dropped from the build:
context, coroutine, coroutine2, graph_parallel, mpi, wave

Proposing dropping these from the SDK build:
atomic, container, exception, graph, math, program_options, random, test, timer

User avatar
adrian_broher
Programmer
Posts: 1072
Joined: Fri Mar 01, 2013 9:52 am
Location: Germany

Re: SDK refactoring - drop some of the Boost library builds

#2 Post by adrian_broher » Mon Jun 05, 2017 1:48 pm

You won't gain much with such a change; out of my head (which does not reflect the latest 1,5 month of development) the result would be:

atomic -- must be kept, as it is an implicit dependency of boost::thread and boost::log
container -- must be kept, as is an implicit dependency of boost::thread
exception -- must be kept, as is an implicit dependency of boost::thread
graph -- must be kept, as is used by universe/Pathfinder.cpp to find fleet routes.
math -- I don't know, but probably some boost internals rely on this and it implements some useful code like quats.
program_options -- can be disabled
random -- must be kept, as it is used by util/Random.h, unless that custom random generator PR was merged (which I hope it wasn't).
test -- must be kept, as it is used by the existing and future automated tests.
timer -- must be kept, as it is used at various places within FO to take time for logging purposes.
Resident code gremlin
Attached patches are released under GPL 2.0 or later.
Git author: Marcel Metz

User avatar
mem359
Dyson Forest
Posts: 214
Joined: Sun Jun 08, 2014 1:18 am

Re: SDK refactoring - drop some of the Boost library builds

#3 Post by mem359 » Mon Jun 05, 2017 2:57 pm

Thanks for your reply. I'll look through the code, in case any of those situations have changed.
adrian_broher wrote: atomic -- must be kept, as it is an implicit dependency of boost::thread and boost::log
container -- must be kept, as is an implicit dependency of boost::thread
exception -- must be kept, as is an implicit dependency of boost::thread
At least for these, I called "otool -L" on the thread and log shared libraries.
Is there something else I should be checking?

I didn't see them depending on atomic, container, or exception.
"thread" needed: system
"log" needed: chrono, date_time, filesystem, regex, thread, and system

I've run (my modified version of) the CMake SDK with all the library builds mentioned dropped.
I don't know (yet) if the libraries are fully functional, but the script completed the build without error.
I would have thought that implicit dependencies would create an error during the build?

User avatar
adrian_broher
Programmer
Posts: 1072
Joined: Fri Mar 01, 2013 9:52 am
Location: Germany

Re: SDK refactoring - drop some of the Boost library builds

#4 Post by adrian_broher » Mon Jun 05, 2017 3:25 pm

mem359 wrote: Is there something else I should be checking?
Grep the boost source code at tag boost-1.59.0 and you will see that the library API headers of said libraries include the corresponding dependencies:

Code: Select all

$ grep -r --exclude-dir test --exclude-dir atomic boost/atomic libs/
libs/thread/include/boost/thread/pthread/once_atomic.hpp:#include <boost/atomic.hpp>
libs/thread/include/boost/thread/executors/detail/scheduled_executor_base.hpp:#include <boost/atomic.hpp>
libs/thread/include/boost/thread/executors/detail/priority_executor_base.hpp:#include <boost/atomic.hpp>
libs/thread/include/boost/thread/concurrent_queues/sync_priority_queue.hpp:#include <boost/atomic.hpp>
libs/thread/include/boost/thread/testable_mutex.hpp:#include <boost/atomic.hpp>
libs/thread/src/pthread/once_atomic.cpp:#include <boost/atomic.hpp>
libs/thread/build/has_atomic_flag_lockfree_test.cpp:#include "../../../boost/atomic.hpp"
libs/thread/build/Jamfile.v2:           result += <library>/boost/atomic//boost_atomic ; 
libs/math/include/boost/math/special_functions/detail/bernoulli_details.hpp:#include <boost/atomic.hpp>
libs/log/include/boost/log/detail/event.hpp:#   include <boost/atomic/capabilities.hpp>
libs/log/include/boost/log/detail/event.hpp:#       include <boost/atomic/atomic.hpp>
libs/log/include/boost/log/detail/event.hpp:#       include <boost/atomic/atomic_flag.hpp>
libs/log/src/event.cpp:#include <boost/atomic/fences.hpp>
libs/lockfree/include/boost/lockfree/detail/atomic.hpp:#include <boost/atomic.hpp>
libs/lockfree/examples/stack.cpp:#include <boost/atomic.hpp>
libs/lockfree/examples/queue.cpp:#include <boost/atomic.hpp>
libs/lockfree/examples/spsc_queue.cpp:#include <boost/atomic.hpp>

Code: Select all

$ grep -r --exclude-dir test --exclude-dir container boost/container libs/
libs/geometry/include/boost/geometry/index/detail/varray_detail.hpp:#include <boost/container/vector.hpp>
libs/geometry/include/boost/geometry/index/detail/varray.hpp:#include <boost/container/detail/config_begin.hpp>
libs/geometry/include/boost/geometry/index/detail/varray.hpp:#include <boost/container/detail/workaround.hpp>
libs/geometry/include/boost/geometry/index/detail/varray.hpp:#include <boost/container/detail/config_end.hpp>
libs/geometry/include/boost/geometry/index/detail/rtree/node/node_elements.hpp:#include <boost/container/vector.hpp>
libs/geometry/include/boost/geometry/index/detail/rtree/node/node.hpp:#include <boost/container/vector.hpp>
libs/thread/include/boost/thread/future.hpp:#include <boost/container/scoped_allocator.hpp>
libs/thread/include/boost/thread/csbl/list.hpp:#include <boost/container/list.hpp>
libs/thread/include/boost/thread/csbl/deque.hpp:#include <boost/container/deque.hpp>
libs/thread/include/boost/thread/csbl/memory/allocator_arg.hpp:#include <boost/container/scoped_allocator.hpp>
libs/thread/include/boost/thread/csbl/memory/scoped_allocator.hpp:#include <boost/container/scoped_allocator.hpp>
libs/thread/include/boost/thread/csbl/memory/allocator_traits.hpp:#include <boost/container/allocator_traits.hpp>
libs/thread/include/boost/thread/csbl/vector.hpp:#include <boost/container/vector.hpp>
libs/thread/include/boost/thread/csbl/queue.hpp:#include <boost/container/queue.hpp>
libs/icl/include/boost/icl/map.hpp:#   include <boost/container/map.hpp>
libs/icl/include/boost/icl/map.hpp:#   include <boost/container/set.hpp>
libs/icl/include/boost/icl/interval_base_set.hpp:#   include <boost/container/set.hpp>
libs/icl/include/boost/icl/set.hpp:#   include <boost/container/set.hpp>
libs/circular_buffer/include/boost/circular_buffer/details.hpp:#include <boost/container/allocator_traits.hpp>
libs/circular_buffer/include/boost/circular_buffer/base.hpp:#include <boost/container/allocator_traits.hpp>
libs/move/example/doc_move_inserter.cpp:#include <boost/container/list.hpp>
libs/move/example/doc_move_iterator.cpp:#include <boost/container/vector.hpp>
libs/move/example/doc_file_descriptor.cpp:#include <boost/container/vector.hpp>
libs/lexical_cast/include/boost/lexical_cast/try_lexical_convert.hpp:#include <boost/container/container_fwd.hpp>
libs/lexical_cast/include/boost/lexical_cast/detail/converter_lexical_streams.hpp:#include <boost/container/container_fwd.hpp>
libs/lexical_cast/include/boost/lexical_cast/detail/converter_lexical.hpp:#include <boost/container/container_fwd.hpp>
libs/lexical_cast/perf/performance_test.cpp:#include <boost/container/string.hpp>
libs/unordered/include/boost/unordered/detail/allocate.hpp:#   include <boost/container/allocator_traits.hpp>
libs/interprocess/include/boost/interprocess/indexes/iunordered_set_index.hpp:#include <boost/container/detail/minimal_char_traits_header.hpp>  //std::char_traits
libs/interprocess/include/boost/interprocess/indexes/iunordered_set_index.hpp:#include <boost/container/detail/placement_new.hpp>
libs/interprocess/include/boost/interprocess/indexes/iset_index.hpp:#include <boost/container/detail/minimal_char_traits_header.hpp>  //std::char_traits
libs/interprocess/include/boost/interprocess/detail/segment_manager_helper.hpp:#include <boost/container/detail/type_traits.hpp> //alignment_of
libs/interprocess/include/boost/interprocess/detail/segment_manager_helper.hpp:#include <boost/container/detail/minimal_char_traits_header.hpp>
libs/interprocess/include/boost/interprocess/detail/transform_iterator.hpp:#include <boost/container/detail/iterator.hpp>
libs/interprocess/include/boost/interprocess/detail/windows_intermodule_singleton.hpp:#include <boost/container/string.hpp>
libs/interprocess/include/boost/interprocess/detail/windows_intermodule_singleton.hpp:#include <boost/container/map.hpp>
libs/interprocess/include/boost/interprocess/detail/intermodule_singleton_common.hpp:#include <boost/container/detail/type_traits.hpp>  //alignment_of, aligned_storage
libs/interprocess/include/boost/interprocess/detail/managed_open_or_create_impl.hpp:#include <boost/container/detail/type_traits.hpp>  //alignment_of, aligned_storage
libs/interprocess/include/boost/interprocess/detail/variadic_templates_tools.hpp:#include <boost/container/detail/variadic_templates_tools.hpp>
libs/interprocess/include/boost/interprocess/detail/named_proxy.hpp:#include <boost/container/detail/placement_new.hpp>
libs/interprocess/include/boost/interprocess/detail/in_place_interface.hpp:#include <boost/container/detail/type_traits.hpp>  //alignment_of, aligned_storage
libs/interprocess/include/boost/interprocess/segment_manager.hpp:#include <boost/container/detail/minimal_char_traits_header.hpp>
libs/interprocess/include/boost/interprocess/segment_manager.hpp:#include <boost/container/detail/placement_new.hpp>
libs/interprocess/include/boost/interprocess/containers/map.hpp:#include <boost/container/map.hpp>
libs/interprocess/include/boost/interprocess/containers/stable_vector.hpp:#include <boost/container/stable_vector.hpp>
libs/interprocess/include/boost/interprocess/containers/slist.hpp:#include <boost/container/slist.hpp>
libs/interprocess/include/boost/interprocess/containers/flat_set.hpp:#include <boost/container/flat_set.hpp>
libs/interprocess/include/boost/interprocess/containers/allocation_type.hpp:#include <boost/container/detail/allocation_type.hpp>
libs/interprocess/include/boost/interprocess/containers/list.hpp:#include <boost/container/list.hpp>
libs/interprocess/include/boost/interprocess/containers/deque.hpp:#include <boost/container/deque.hpp>
libs/interprocess/include/boost/interprocess/containers/vector.hpp:#include <boost/container/vector.hpp>
libs/interprocess/include/boost/interprocess/containers/string.hpp:#include <boost/container/string.hpp>
libs/interprocess/include/boost/interprocess/containers/version_type.hpp:#include <boost/container/detail/version_type.hpp>
libs/interprocess/include/boost/interprocess/containers/containers_fwd.hpp:#include <boost/container/container_fwd.hpp>
libs/interprocess/include/boost/interprocess/containers/set.hpp:#include <boost/container/set.hpp>
libs/interprocess/include/boost/interprocess/containers/pair.hpp:#include <boost/container/detail/pair.hpp>
libs/interprocess/include/boost/interprocess/containers/flat_map.hpp:#include <boost/container/flat_map.hpp>
libs/interprocess/include/boost/interprocess/smart_ptr/detail/shared_count.hpp:#include <boost/container/allocator_traits.hpp>
libs/interprocess/include/boost/interprocess/smart_ptr/detail/shared_count.hpp:#include <boost/container/detail/placement_new.hpp>
libs/interprocess/include/boost/interprocess/smart_ptr/detail/sp_counted_impl.hpp:#include <boost/container/allocator_traits.hpp>
libs/interprocess/include/boost/interprocess/mem_algo/detail/simple_seq_fit_impl.hpp:#include <boost/container/detail/multiallocation_chain.hpp>
libs/interprocess/include/boost/interprocess/mem_algo/detail/mem_algo_common.hpp:#include <boost/container/detail/multiallocation_chain.hpp>
libs/interprocess/include/boost/interprocess/mem_algo/detail/mem_algo_common.hpp:#include <boost/container/detail/placement_new.hpp>
libs/interprocess/include/boost/interprocess/mem_algo/rbtree_best_fit.hpp:#include <boost/container/detail/multiallocation_chain.hpp>
libs/interprocess/include/boost/interprocess/mem_algo/rbtree_best_fit.hpp:#include <boost/container/detail/placement_new.hpp>
libs/interprocess/include/boost/interprocess/offset_ptr.hpp:#include <boost/container/detail/type_traits.hpp>  //alignment_of, aligned_storage
libs/interprocess/include/boost/interprocess/allocators/adaptive_pool.hpp:#include <boost/container/detail/multiallocation_chain.hpp>
libs/interprocess/include/boost/interprocess/allocators/private_adaptive_pool.hpp:#include <boost/container/detail/multiallocation_chain.hpp>
libs/interprocess/include/boost/interprocess/allocators/allocator.hpp:#include <boost/container/detail/multiallocation_chain.hpp>
libs/interprocess/include/boost/interprocess/allocators/allocator.hpp:#include <boost/container/detail/placement_new.hpp>
libs/interprocess/include/boost/interprocess/allocators/detail/allocator_common.hpp:#include <boost/container/detail/multiallocation_chain.hpp>
libs/interprocess/include/boost/interprocess/allocators/detail/allocator_common.hpp:#include <boost/container/detail/placement_new.hpp>
libs/interprocess/include/boost/interprocess/allocators/detail/adaptive_node_pool.hpp:#include <boost/container/detail/adaptive_node_pool_impl.hpp>
libs/interprocess/include/boost/interprocess/allocators/detail/node_pool.hpp:#include <boost/container/detail/node_pool_impl.hpp>
libs/interprocess/include/boost/interprocess/allocators/private_node_allocator.hpp:#include <boost/container/detail/multiallocation_chain.hpp>
libs/interprocess/include/boost/interprocess/allocators/node_allocator.hpp:#include <boost/container/detail/multiallocation_chain.hpp>
libs/interprocess/include/boost/interprocess/sync/windows/sync_utils.hpp:#include <boost/container/map.hpp>
libs/interprocess/include/boost/interprocess/sync/shm/named_creation_functor.hpp:#include <boost/container/detail/placement_new.hpp>

Code: Select all

$ grep -r --exclude *.patch --exclude-dir test --exclude-dir doc --exclude-dir exception boost/exception libs/
libs/thread/include/boost/thread/future.hpp:#include <boost/exception_ptr.hpp>
libs/thread/include/boost/thread/experimental/parallel/v1/exception_list.hpp:#include <boost/exception_ptr.hpp>
libs/thread/include/boost/thread/exceptional_ptr.hpp:#include <boost/exception_ptr.hpp>
libs/xpressive/include/boost/xpressive/regex_error.hpp:#include <boost/exception/exception.hpp>
libs/xpressive/include/boost/xpressive/regex_error.hpp:#include <boost/exception/info.hpp>
libs/context/example/rethrow.cpp:#include <boost/exception_ptr.hpp>
libs/context/example/exception.cpp:#include <boost/exception_ptr.hpp>
libs/throw_exception/include/boost/throw_exception.hpp:# include <boost/exception/exception.hpp>
libs/algorithm/include/boost/algorithm/hex.hpp:#include <boost/exception/all.hpp>
libs/log/include/boost/log/support/exception.hpp:#include <boost/exception/info.hpp>
libs/log/src/exceptions.cpp:#include <boost/exception/exception.hpp>
libs/log/src/exceptions.cpp:#include <boost/exception/errinfo_at_line.hpp>
libs/log/src/exceptions.cpp:#include <boost/exception/info.hpp>
libs/log/src/format_parser.cpp:#include <boost/exception/exception.hpp>
libs/log/src/unhandled_exception_count.cpp: * https://github.com/panaseleus/stack_unwinding/blob/master/boost/exception/uncaught_exception_count.hpp
libs/log/src/settings_parser.cpp:#include <boost/exception/exception.hpp>
libs/log/src/settings_parser.cpp:#include <boost/exception/info.hpp>
libs/log/src/settings_parser.cpp:#include <boost/exception/errinfo_at_line.hpp>
libs/coroutine/include/boost/coroutine/detail/pull_coroutine_object.hpp:#include <boost/exception_ptr.hpp>
libs/coroutine/include/boost/coroutine/detail/pull_coroutine_impl.hpp:#include <boost/exception_ptr.hpp>
libs/coroutine/include/boost/coroutine/detail/push_coroutine_impl.hpp:#include <boost/exception_ptr.hpp>
libs/coroutine/include/boost/coroutine/detail/push_coroutine_object.hpp:#include <boost/exception_ptr.hpp>
libs/coroutine/include/boost/coroutine/detail/trampoline_push.hpp:#include <boost/exception_ptr.hpp>
I've run (my modified version of) the CMake SDK with all the library builds mentioned dropped.
I don't know (yet) if the libraries are fully functional, but the script completed the build without error.
Then it is not a viable benchmark for checking if the libraries are required or not. Unless the build sucessfully builds on all platforms that consume the SDK.
I would have thought that implicit dependencies would create an error during the build?
No, especially if the dependencies are within the headers only. Unless the headers are included during the build into a compilation unit and the compiler can't locate the missing header the SDK build is not able to tell you that it is incomplete. Only building freeorion itself can tell you that.
Resident code gremlin
Attached patches are released under GPL 2.0 or later.
Git author: Marcel Metz

User avatar
mem359
Dyson Forest
Posts: 214
Joined: Sun Jun 08, 2014 1:18 am

Re: SDK refactoring - drop some of the Boost library builds

#5 Post by mem359 » Mon Jun 05, 2017 4:23 pm

adrian_broher wrote: No, especially if the dependencies are within the headers only. Unless the headers are included during the build into a compilation unit and the compiler can't locate the missing header the SDK build is not able to tell you that it is incomplete. Only building freeorion itself can tell you that.
Ah, thanks.
I forgot that all of the include files are unpacked and available, even if the library building is skipped.

Request withdrawn.

You mentioned that we might be able to drop program_options or math, but I will try the entire path (SDK through running FO) as a check first.

Post Reply