Development: Create deployment workflow for the new microservices architecture

Motivation and Context

This PR creates a workflow for the deployment of the microservices architecture on the new artemitest6 server. It also creates a docker-compose file for the microservices components.
Review Progress

Code Review

[ ] Review 1
[ ] Review 2

Thank you for your feedback @jpbernius. I implemented the changes suggested. Can you have a look again?

Thank you for the feedback @Mtze! I applied your suggestions, I hope it is all good now

The problem was because I was building the docker images using always the Artemis.war. I also updated the dockerfiles and some other inconsistencies in the docker compose file

LGTM - As a working prototype this is fine. The deployment to artemistest6 was successful.
CONTAINER ID   IMAGE                                                     COMMAND                   CREATED        STATUS        PORTS                                                                                NAMES
9ee92265f0d1   ghcr.io/ls1intum/artemis-service:pr-4329                  "java -jar Artemis.w…"    24 hours ago   Up 24 hours   8080/tcp                                                                                artemis_artemis-app_1
cf246e81d1b8   ghcr.io/ls1intum/artemis-usermanagement-service:pr-4329   "java -jar user-mana…"    24 hours ago   Up 24 hours   8087/tcp                                                                                artemis_usermanagement-app_1
1d9e204875e5   ghcr.io/ls1intum/artemis-lecture-service:pr-4329          "java -jar lecture.w…"    24 hours ago   Up 24 hours   8086/tcp                                                                                artemis_lecture-app_1
9e5d3a144e3b   ghcr.io/ls1intum/artemis-gateway:pr-4329                  "java -jar gateway.w…"    24 hours ago   Up 24 hours   <IP_ADDRESS>:8089->8089/tcp, :::8089->8089/tcp                                                                 artemis_gateway-app_1
3a3244958afd   vromero/activemq-artemis:latest                           "/docker-entrypoint.…"    24 hours ago   Up 24 hours   1883/tcp, 5445/tcp, 5672/tcp, 9404/tcp, 61613/tcp, 61616/tcp, <IP_ADDRESS>:8161->8161/tcp, :::8161->8161/tcp   artemis_activemq-broker_1
0cb0f7692179   mysql:8.0.27                                              "docker-entrypoint.s…"    24 hours ago   Up 24 hours   <IP_ADDRESS>:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp                                                      artemis_mysql
b10886bd8c5a   jhipster/jhipster-registry:v6.8.0                         "/bin/sh -c 'echo \"T…"   24 hours ago   Up 24 hours   <IP_ADDRESS>:8761->8761/tcp, :::8761->8761/tcp                                                                 artemis_jhipster-registry_1

However, for production use we have to address the following issues (most likely non-exhaustive):

Configuration Files - We should definitely try to move all the configuration properties into the docker environment. (eg. jira account username/password). Managing the necessary configuration files is tedious and error prone. Especially for future micro services this will complicate things.
Documentation - We need to provide good documentation for the Microservices setup (how to setup, how to update, what has to taken into account for new micro services, etc)
Persistent Storage - In the current state, data is not stored persistently. Each time the containers are recreated all user data is lost. We need to add volumes for each "data-creating" service.

Minor concern: The database is exposed outside of the docker network. This is great for debugging purposes, but a database must not be exposed to the internet in a production environment. In our case this is not to critical because we block all traffic expect ports 80 and 443 on the network side. However this has to be documented somewhere (or even disabled in the prod override to only expose the mysql port inside the docker network)

well, I am not experienced with the deployment workflows, I looked at the changes to see if there is anything that seems weird or causes a problem. I think it looks good, also testserver6 seems fine.


Exposes PostPurchaseRenderApi type from post-purchase-ui-extensions

Background
Currently, any code using post-purchase-ui-extensions and post-purchase-ui-extensions-react have to do a verbose filepath import in order to access the PostPurchaseRenderApi type , rather than a cleaner top-level export:
import {PostPurchaseRenderApi} from '@shopify/post-purchase-ui-extensions/src/extension-points/api';

This also was an issue before the Argo rename:
import {PostPurchaseRenderApi} from '@shopify/argo-post-purchase/src/extension-points/api';

Ideally, the import could be just be succinct like this, similar to all other type and member imports from ui-extensions:
import {PostPurchaseRenderApi} from '@shopify/post-purchase-ui-extensions';
// OR
import {PostPurchaseRenderApi} from '@shopify/post-purchase-ui-extensions-react';

Solution
I simply had to add PostPurchaseRenderApi to some export statements in Barrel Files.
🎩
I'm not personally sure how to view the build output after this change, I would like some pointers on this from reviewers. After we determine that, I would inspect the .d.ts output from a build to ensure that PostPurchaseRenderApi is present as a top-level export.
Checklist

[ ] I have :tophat:'d these changes

/cc @alxclark


RuntimeError: CUDA error: device-side assert triggered

I ran into this error when I ran the code, I'm a noob, I would like to ask if there are any big guys who have encountered this problem and solved it

Please carefully check whether cuda and torch are correctly installed.


Add localhost as default network

Steps to reproduce
Only hardhat is displayed when selecting "Run scripts" if no network has been set up.
Expected behaviour
Hardhat localhost should also be a default choice
Actual behavior
Currently only hardhat is default

Includes in #64


[QA] add phpstan

Configuration in the README (https://github.com/phpstan/phpstan).

cc @ondrejmirtes

@f3l1x I can prepare a PR if you'd like me to.

@ondrejmirtes Thanks a lot, but it not nescessary. I would like to go throw the readme and handle possible breakpoints by my self. I can provide some feedback after that.
Thank you for your jof on @phpstan. It looks like a great missing tool for PHP world. :+1:

@f3l1x OK, just show me the integration before merging so I can give my 2 cents on it :)

@ondrejmirtes I've finally integrated it. :-) Good job btw, phpstan's hits are pretty useful and I didn't get lost.
I would be happy if you could take a look.

@f3l1x Cool, it's nice :) 👍 Are you still extending Nette\Object for BC reasons?

@ondrejmirtes Well, I do not, but nextras/orm does, so I cannot avoid it.


feat: update diagram

Type of PR

Documentation changes

Purpose

Update Fabric E2E Architecture

Does this introduce a breaking change? If yes, details on what can break
Author pre-publish checklist

[ ] Added test to prove my fix is effective or new feature works
[X] No PII in logs
[ ] Made corresponding changes to the documentation

Validation steps

...

Issues Closed or Referenced

References #788

Using Copy + SQL Proc instead of Spark notebooks to copy data into Warehouse. Essentially, Copy data into "interim" tables in the warehouse, then call a Stored Proc to transform into Star Schema.
Alternatives tried:

Spark Notebook - no support for landing data directly to a warehouse
TSQL Notebook - currently no support for notebook parameters. No support for Git and Deployment pipelines.


Cleaning large amount of images

Hi, I had to run the state machine multiple times, to clean all the ECR images.
Even though the output stated it e.g. deleted 300+ images, only a fraction of the images got deleted.
I guess it's either due to the 100 id length limit on the ECR batch delete command (https://docs.aws.amazon.com/AmazonECR/latest/APIReference/API_BatchDeleteImage.html#API_BatchDeleteImage_RequestSyntax)
or
maybe it makes sense to not silently increase the deleted counter (https://github.com/jogold/cloudstructs/blob/master/src/toolkit-cleaner/clean-images.lambda.ts#L47), but instead inspect the response of the batch delete command, seems like it can also contain information about both successful and failed image ids (https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-client-ecr/Interface/BatchDeleteImageCommandOutput/)
anyway, very useful so far, I will just run it multiple times.

Hi @OG84
This is strange because I use DescribeImages which returns up to 100 images. The results are filtered and then passed to BatchDeleteImage.

yeah you're right, then it must be something else on my side.


#0: Update models/demos subdirectory code owners

The models/demos subdirectories had one owner. Add more owners.

@bbradelTT has proposed having more owners of javelin models to ensure changes can go in if people are not available.
For example, @yieldthought is at a conference and @uaydonat is on vacation.
cc: @jvasilje @uaydonat

Yeah, we need at least two people per slot.   I like the idea that model leads own these directories.

@tt-rkim the owners of the non-model lines are:
SeanNijjar
aliuTT
mo-tenstorrent
pgkeller
rtawfik01
tapspatel
tt-aho
tt-nshanker
tt-rkim
ubcheema

It would be good if you updated those lines in a subsequent PR since you have a better idea about who to add as secondaries for those files.

@bbradelTT By the way in case you weren't aware, these changes will also force owners who were originally alone to now have to get approval from others. Co-owners cannot change files at will. Just in case you thought that was the case

@bbradelTT By the way in case you weren't aware, these changes will also force owners who were originally alone to now have to get approval from others. Co-owners cannot change files at will. Just in case you thought that was the case

@tt-rkim That's frustrating. Unless there are three people per line having only two people would cause the same problem as before, except for people that will most likely make changes. I guess there are no plans to change that, right?

@mtairum should be added to mixtral.
@xuncaiTT should be added to llama*
yes, having multiple approvers makes sense if people are away to keep things moving. However, unless the main owners (first in the list) are away (or submitting code themselves), they should be in charge of the approvals.

@tt-rkim @xuncaiTT
I updated the file based on the newest comment and I'm getting the following error:
Unknown owner on line 161: make sure @xuncaiTT exists and has write access to the repository
@xuncaiTT does exist and seems to have commits to the repository.
Any idea what's going on?

@bbradelTT Pls use @caixunshiren to specify Jack instead of @xuncaiTT
He currently uses the former account.
@uaydonat FYI

@djordje-tt should also be in models/demos/wormhole/falcon7b


Itinerary Details Page - Add Team Member AJAX

If you add a team member to the team for an itinerary, the team lead dropdown is not shown/updated without a full page reload - this is because some of the page functions via ajax/js code. We need to either refactor to update the knockout model so we can show/hide the team lead section as part of the ajax result  or remove the ajax code on this page.

@stevejgordon @tonysurma
Since I've been working on some of the Itinerary related issues.  I decided to take a look at this one as well 😄

Thanks @c0g1t8 - I'm inclined to suggest we factor out the ajax and knockout here. While it provides a nice no postback style update, it does increase our barrier to entry. It's been removed in most other places.

@stevejgordon  I've factored out the ajax and knockout as you suggested.  This also required a little reworking of the associated view.  Testing it now.

@stevejgordon Looking at your implementation of RemoveTeamLead and SetTeamLead in the ItineraryController class.  Noted your comments in the RemoveTeamLead method and am using it as a pattern for AddTeamMember.

Need to review this fix once #1864 is closed.


KafkaStreamsStateStore not working when the store value is an Avro SpecificRecord

The context for this issue can be gotten at this SO question, and was discussed with @sobychacko: https://stackoverflow.com/questions/56634598/kafkastreamsstatestore-not-working-when-the-store-value-is-an-avro-specificrecor
When dealing with Avro messages in KStream applications, Spring Cloud Stream doesn't rely on its own SerDes, but requires using the ones provided by confluent, e.g:
    @StreamListener
    @SendTo("output")
    fun process(@Input("input") input: KStream<Int, Customer>, @Input("order") orderEvent: KStream<Int, GenericRecord>): KStream<Int, OrderShippedEvent> {

        val serdeConfig = mapOf(
                AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG to "http://localhost:8081")

        val intSerde = Serdes.IntegerSerde()
        val customerSerde = SpecificAvroSerde<Customer>()
        customerSerde.configure(serdeConfig, false)
        val orderCreatedSerde = SpecificAvroSerde<OrderCreatedEvent>()
        orderCreatedSerde.configure(serdeConfig, false)
        val orderShippedSerde = SpecificAvroSerde<OrderShippedEvent>()
        orderShippedSerde.configure(serdeConfig, false)

        val stateStore: Materialized<Int, Customer, KeyValueStore<Bytes, ByteArray>> =
                Materialized.`as`<Int, Customer, KeyValueStore<Bytes, ByteArray>>("customer-store")
                        .withKeySerde(intSerde)
                        .withValueSerde(customerSerde)

In the previous example I also had registered a ConfluentSchemaRegistry bean, that addressed deserialization of the input kstream, but inside the StreamListener I had to rely on Confluent's SpecificAvroSerde, which is kind of redundant, but allows us to do what we want.
However, in the case of KafkaStreamsStateStore, there's no way to serialize a keystore where the value is an Avro type.
Supposing I had this StreamListener:
    @KafkaStreamsStateStore(name = DEDUP_STORE, type = KafkaStreamsStateStoreProperties.StoreType.KEYVALUE,
            valueSerde = "io.confluent.kafka.streams.serdes.avro.SpecificAvroSerde")
    @StreamListener
    @SendTo("output")
    fun process(@Input("input") input: KStream<String, TransferEmitted>): KStream<String, TransferEmitted> {
        val serdeConfig = mapOf(AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG to "http://localhost:8081")
        return input
                .transform(TransformerSupplier { DeduplicationTransformer() }, DEDUP_STORE)
                .filter { _, value -> fraudDetectionService.isFraudulent(value) }
    }

I set up a value serde of type SpecificAvroSerde and this class has two constructors, a no-arg, and one that receives a SchemaRegistryClient. KeyValueSerdeResolver.getValueSerde instantiantes the serde without using the schema registry client, so it eventually fails, since it's null:
private Serde<?> getValueSerde(String valueSerdeString)
			throws ClassNotFoundException {
		Serde<?> valueSerde;
		if (StringUtils.hasText(valueSerdeString)) {
			valueSerde = Utils.newInstance(valueSerdeString, Serde.class);
		}

I thought we could work around this by passing the SchemaRegistryClient  bean into the other SpecificAvroRecord constructor.
This would require modifying these classes, passing the registry client bean:
KafkaStreamsBinderSupportAutoConfiguration
    @Bean
    @SuppressWarnings("unchecked")
    @ConditionalOnMissingBean
    public KeyValueSerdeResolver keyValueSerdeResolver(
            @Qualifier("streamConfigGlobalProperties") Object streamConfigGlobalProperties,
            KafkaStreamsBinderConfigurationProperties properties,
            Optional<SchemaRegistryClient> schemaRegistryClient) {
        if(schemaRegistryClient.isPresent()) {
            return new KeyValueSerdeResolver(
                    (Map<String, Object>) streamConfigGlobalProperties, properties, schemaRegistryClient.get());
        } else {
            return new KeyValueSerdeResolver(
                    (Map<String, Object>) streamConfigGlobalProperties, properties);
        }
    }

KeyValueSerdeResolver
    KeyValueSerdeResolver(Map<String, Object> streamConfigGlobalProperties,
                          KafkaStreamsBinderConfigurationProperties binderConfigurationProperties) {
        this.streamConfigGlobalProperties = streamConfigGlobalProperties;
        this.binderConfigurationProperties = binderConfigurationProperties;
        this.schemaRegistryClient = null;
    }
...
private Serde<?> getValueSerde(String valueSerdeString)
            throws ClassNotFoundException {
        Serde<?> valueSerde;
        if (StringUtils.hasText(valueSerdeString)) {
            if("io.confluent.kafka.streams.serdes.avro.SpecificAvroSerde".equals(valueSerdeString)) {
                valueSerde = Utils.newParameterizedInstance(valueSerdeString, SchemaRegistryClient.class, schemaRegistryClient);
            }else {
                valueSerde = Utils.newInstance(valueSerdeString, Serde.class);
            }
        }

However this is really naive, because SpecificAvroSerde is expecting an io.confluent.kafka.schemaregistry.client.SchemaRegistryClient in the constructor, not Spring's org.springframework.cloud.stream.schema.client.SchemaRegistryClient.
So, my initial idea isn't valid... Offhand I can think of other approaches, such as providing Spring's own Avro SerDes that take into account Spring's SchemaRegistryClient, or maybe creating the confluent SpecificAvroSerde manually, not invoking the constructor through reflection, something like:
KeyValueSerdeResolver.getValueSerde:
    private Serde<?> getValueSerde(String valueSerdeString)
            throws ClassNotFoundException {
        Serde<?> valueSerde;
        if (StringUtils.hasText(valueSerdeString)) {
            if("io.confluent.kafka.streams.serdes.avro.SpecificAvroSerde".equals(valueSerdeString)) {
                val serdeConfig = mapOf(
                AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG to registryUrl)
                valueSerde = SpecificAvroSerde<xxx>()
                valueSerde.configure(serdeConfig, false)
            }

@codependent Thank you for this detailed report. Here is another idea that we can consider. How about the application creates a bean of type StoreBuilder and then the StreamListener processor injects the store when it builds the StreamsBuilder? That way, the application can bring whatever dependencies the Serde requires. This avoids the need for the binder to include dependencies to Confluent Serde classes explicitly or through reflection.
3.0 version of the binder (basically, the current master) embraces the functional model of writing Kafka Streams application fully using the java.util.function.Function approach. The initial support for this is already introduced in 2.2.0 (Germantown). In this model, it does not support the usage of KafkaStreamsStateStore annotation at all and requires the strategy I just mentioned above. Here, you can find such an approach. Although StreamListener doesn't support this currently, we can introduce it there as well. In fact, I was thinking to deprecate the use of KafkaStreamsStateStore annotation in 3.0, in favor of the functional model and the state store beans.
We can certainly consider this approach for StreamListener in 3.0. What do you think?

That will definitely do and that sounds way better that what I said. It'll provide the flexibility we need to configure the StateStore tweaking the Serdes as needed.
Will it be available starting with Horsham or are you planning to backport it to Germantownas well?

We can definitely add that as part of StreamListener in Horsham. I wasn't thinking to backport that to Germantown, but might not be a bad idea to incorporate that with StreamListener there.


Minor fixes and suggestions

I couldn't really see the diffs very well but went ahead and merged it on trust.


Stack smashing detected

@damonkohler This is the same issue that I sent over email, opening a ticket with more detail.
Platform: Ubuntu 16.04, ROS Kinetic, x86_64
When I try and run the turtlebot launch (off of the latest version of this repo) I get a
stack smashing error, and the program SIGABRTs.
I tried to debug this by running it in GCC's ASAN (https://github.com/google/sanitizers/wiki/AddressSanitizer), but then everything works.
(maybe it is a race condidtion being covered up because of ASANs overhead?)
I ran it in gdb to get this backtrace:
#0  0x00007ffff4b17418 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff4b1901a in __GI_abort () at abort.c:89
#2  0x00007ffff4b5972a in __libc_message (do_abort=do_abort@entry=1, fmt=fmt@entry=0x7ffff4c70c7f "*** %s ***: %s terminated\n") at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007ffff4bfa89c in __GI___fortify_fail (msg=<optimized out>, msg@entry=0x7ffff4c70c61 "stack smashing detected") at fortify_fail.c:37
#4  0x00007ffff4bfa840 in __stack_chk_fail () at stack_chk_fail.c:28
#5  0x00007ffff7984596 in ceres::internal::CovarianceImpl::ComputeCovarianceValuesUsingSuiteSparseQR() () from /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1
#6  0x00007ffff7989ea5 in ceres::internal::CovarianceImpl::ComputeCovarianceValues() () from /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1
#7  0x00007ffff7989f59 in ceres::internal::CovarianceImpl::Compute(std::vector<std::pair<double const*, double const*>, std::allocator<std::pair<double const*, double const*> > > const&, ceres::internal::ProblemImpl*) ()
   from /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1
#8  0x000000000061832a in cartographer::mapping_2d::scan_matching::CeresScanMatcher::Match(cartographer::transform::Rigid2<double> const&, cartographer::transform::Rigid2<double> const&, std::vector<Eigen::Matrix<float, 2, 1, 0, 2, 1>, std::allocator<Eigen::Matrix<float, 2, 1, 0, 2, 1> > > const&, cartographer::mapping_2d::ProbabilityGrid const&, cartographer::transform::Rigid2<double>*, Eigen::Matrix<double, 3, 3, 0, 3, 3>*, ceres::Solver::Summary*) const ()
#9  0x0000000000601df8 in cartographer::mapping_2d::LocalTrajectoryBuilder::ScanMatch(std::chrono::time_point<cartographer::common::UniversalTimeScaleClock, std::chrono::duration<long, std::ratio<1l, 10000000l> > >, cartographer::transform::Rigid3<double> const&, cartographer::transform::Rigid3<double> const&, cartographer::sensor::LaserFan const&, cartographer::transform::Rigid3<double>*, Eigen::Matrix<double, 6, 6, 0, 6, 6>*) ()
#10 0x0000000000602454 in cartographer::mapping_2d::LocalTrajectoryBuilder::AddHorizontalLaserFan(std::chrono::time_point<cartographer::common::UniversalTimeScaleClock, std::chrono::duration<long, std::ratio<1l, 10000000l> > >, cartographer::sensor::LaserFan3D const&) ()
#11 0x0000000000605197 in cartographer::mapping_2d::GlobalTrajectoryBuilder::AddHorizontalLaserFan(std::chrono::time_point<cartographer::common::UniversalTimeScaleClock, std::chrono::duration<long, std::ratio<1l, 10000000l> > >, cartographer::sensor::LaserFan3D const&) ()
#12 0x000000000053bf9f in cartographer_ros::(anonymous namespace)::Node::AddHorizontalLaserFan(long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cartographer::sensor::proto::LaserScan const&) ()
#13 0x000000000053f4b1 in cartographer_ros::(anonymous namespace)::Node::HandleSensorData(long, std::unique_ptr<cartographer_ros::(anonymous namespace)::SensorData, std::default_delete<cartographer_ros::(anonymous namespace)::SensorData> >) ()
#14 0x000000000053c800 in cartographer_ros::(anonymous namespace)::Node::Initialize()::{lambda(long, std::unique_ptr<cartographer_ros::(anonymous namespace)::SensorData, std::default_delete<cartographer_ros::(anonymous namespace)::SensorData> >)#2}::operator()(long, std::unique_ptr<cartographer_ros::(anonymous namespace)::SensorData, std::default_delete<cartographer_ros::(anonymous namespace)::SensorData> >) const ()
#15 0x000000000054263b in std::_Function_handler<void (long, std::unique_ptr<cartographer_ros::(anonymous namespace)::SensorData, std::default_delete<cartographer_ros::(anonymous namespace)::SensorData> >), cartographer_ros::(anonymous namespace)::Node::Initialize()::{lambda(long, std::unique_ptr<cartographer_ros::(anonymous namespace)::SensorData, std::default_delete<cartographer_ros::(anonymous namespace)::SensorData> >)#2}>::_M_invoke(std::_Any_data const&, long&&, std::unique_ptr<cartographer_ros::(anonymous namespace)::SensorData, std::default_delete<cartographer_ros::(anonymous namespace)::SensorData> >&&) ()
#16 0x00000000005427cb in std::function<void (long, std::unique_ptr<cartographer_ros::(anonymous namespace)::SensorData, std::default_delete<cartographer_ros::(anonymous namespace)::SensorData> >)>::operator()(long, std::unique_ptr<cartographer_ros::(anonymous namespace)::SensorData, std::default_delete<cartographer_ros::(anonymous namespace)::SensorData> >) const ()
#17 0x0000000000541646 in cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::AddTrajectory(int, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (long, std::unique_ptr<cartographer_ros::(anonymous namespace)::SensorData, std::default_delete<cartographer_ros::(anonymous namespace)::SensorData> >)>)::{lambda(std::unique_ptr<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value, std::default_delete<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value> >)#1}::operator()(std::unique_ptr<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value, std::default_delete<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value> >) const ()
#18 0x0000000000543d52 in std::_Function_handler<void (std::unique_ptr<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value, std::default_delete<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value> >), cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::AddTrajectory(int, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::function<void (long, std::unique_ptr<cartographer_ros::(anonymous namespace)::SensorData, std::default_delete<cartographer_ros::(anonymous namespace)::SensorData> >)>)::{lambda(std::unique_ptr<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value, std::default_delete<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value> >)#1}>::_M_invoke(std::_Any_data const&, std::unique_ptr<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value, std::default_delete<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value> >&&) ()
#19 0x0000000000544a2d in std::function<void (std::unique_ptr<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value, std::default_delete<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value> >)>::operator()(std::unique_ptr<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value, std::default_delete<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value> >) const ()
#20 0x0000000000543719 in cartographer::common::OrderedMultiQueue<cartographer::mapping::SensorCollatorQueueKey, long, cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value>::Dispatch() ()
#21 0x00000000005421aa in cartographer::common::OrderedMultiQueue<cartographer::mapping::SensorCollatorQueueKey, long, cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value>::Add(cartographer::mapping::SensorCollatorQueueKey const&, long const&, std::unique_ptr<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value, std::default_delete<cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::Value> >) ()
#22 0x0000000000540652 in cartographer::mapping::SensorCollator<cartographer_ros::(anonymous namespace)::SensorData>::AddSensorData(int, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<cartographer_ros::(anonymous namespace)::SensorData, std::default_delete<cartographer_ros::(anonymous namespace)::SensorData> >) ()
#23 0x000000000053bdae in cartographer_ros::(anonymous namespace)::Node::LaserScanMessageCallback(boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&) ()
#24 0x000000000054801c in boost::_mfi::mf1<void, cartographer_ros::(anonymous namespace)::Node, boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&>::operator()(cartographer_ros::(anonymous namespace)::Node*, boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&) const ()
#25 0x000000000054777d in void boost::_bi::list2<boost::_bi::value<cartographer_ros::(anonymous namespace)::Node*>, boost::arg<1> >::operator()<boost::_mfi::mf1<void, cartographer_ros::(anonymous namespace)::Node, boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&>, boost::_bi::list1<boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&> >(boost::_bi::type<void>, boost::_mfi::mf1<void, cartographer_ros::(anonymous namespace)::Node, boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&>&, boost::_bi::list1<boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&>&, int) ()
#26 0x000000000054694d in void boost::_bi::bind_t<void, boost::_mfi::mf1<void, cartographer_ros::(anonymous namespace)::Node, boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&>, boost::_bi::list2<boost::_bi::value<cartographer_ros::(anonymous namespace)::Node*>, boost::arg<1> > >::operator()<boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&>(boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&) ()
#27 0x00000000005458e9 in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, cartographer_ros::(anonymous namespace)::Node, boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&>, boost::_bi::list2<boost::_bi::value<cartographer_ros::(anonymous namespace)::Node*>, boost::arg<1> > >, void, boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&>::invoke(boost::detail::function::function_buffer&, boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&) ()
#28 0x00000000005738fd in boost::function1<void, boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&>::operator()(boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&) const ()
#29 0x000000000056fbd1 in boost::detail::function::void_function_obj_invoker1<boost::function<void (boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&)>, void, boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> >::invoke(boost::detail::function::function_buffer&, boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const>) ()
#30 0x000000000057fe46 in boost::function1<void, boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> >::operator()(boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const>) const ()
#31 0x000000000057dae9 in ros::SubscriptionCallbackHelperT<boost::shared_ptr<sensor_msgs::LaserScan_<std::allocator<void> > const> const&, void>::call(ros::SubscriptionCallbackHelperCallParams&) ()
#32 0x00007ffff63cf25d in ros::SubscriptionQueue::call() () from /opt/ros/kinetic/lib/libroscpp.so
#33 0x00007ffff6379880 in ros::CallbackQueue::callOneCB(ros::CallbackQueue::TLS*) () from /opt/ros/kinetic/lib/libroscpp.so
#34 0x00007ffff637ac83 in ros::CallbackQueue::callAvailable(ros::WallDuration) () from /opt/ros/kinetic/lib/libroscpp.so
#35 0x00007ffff63d29b9 in ros::SingleThreadedSpinner::spin(ros::CallbackQueue*) () from /opt/ros/kinetic/lib/libroscpp.so
#36 0x00007ffff63b83cb in ros::spin() () from /opt/ros/kinetic/lib/libroscpp.so
#37 0x000000000053f553 in cartographer_ros::(anonymous namespace)::Node::SpinForever() ()
#38 0x000000000053f6eb in cartographer_ros::(anonymous namespace)::Run() ()
#39 0x000000000054011a in main ()

If I disable the stack protector by adding -o overflow -fno-stack-protector to the compiler flags of ceres and cartographer_ros, it runs for 30sec - 1min before segfaulting.
(gdb) backtrace
#0  0x00007ffff7a2efcf in ceres::internal::ReorderProgramForSparseNormalCholesky(ceres::SparseLinearAlgebraLibraryType, ceres::OrderedGroups<double*> const&, ceres::internal::Program*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) () from /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1
#1  0x00007ffff7a4e2dd in ceres::internal::TrustRegionPreprocessor::Preprocess(ceres::Solver::Options const&, ceres::internal::ProblemImpl*, ceres::internal::PreprocessedProblem*) ()
   from /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1
#2  0x00007ffff7a463f4 in ceres::Solver::Solve(ceres::Solver::Options const&, ceres::Problem*, ceres::Solver::Summary*) () from /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1
#3  0x00007ffff7a46edf in ceres::Solve(ceres::Solver::Options const&, ceres::Problem*, ceres::Solver::Summary*) () from /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1
#4  0x0000000000601505 in cartographer::mapping_2d::sparse_pose_graph::OptimizationProblem::Solve(std::vector<cartographer::mapping::SparsePoseGraph::Constraint2D, std::allocator<cartographer::mapping::SparsePoseGraph::Constraint2D> > const&, std::vector<cartographer::mapping::Submaps const*, std::allocator<cartographer::mapping::Submaps const*> > const&, std::vector<cartographer::transform::Rigid2<double>, std::allocator<cartographer::transform::Rigid2<double> > > const&, std::vector<cartographer::transform::Rigid2<double>, std::allocator<cartographer::transform::Rigid2<double> > >*, std::vector<cartographer::transform::Rigid2<double>, std::allocator<cartographer::transform::Rigid2<double> > >*) ()
#5  0x00000000005f69a9 in cartographer::mapping_2d::SparsePoseGraph::RunOptimization() ()
#6  0x00000000005f752d in std::_Function_handler<void (std::vector<cartographer::mapping::SparsePoseGraph::Constraint2D, std::allocator<cartographer::mapping::SparsePoseGraph::Constraint2D> > const&), cartographer::mapping_2d::SparsePoseGraph::HandleScanQueue()::{lambda(std::vector<cartographer::mapping::SparsePoseGraph::Constraint2D, std::allocator<cartographer::mapping::SparsePoseGraph::Constraint2D> > const&)#1}>::_M_invoke(std::_Any_data const&, std::vector<cartographer::mapping::SparsePoseGraph::Constraint2D, std::allocator<cartographer::mapping::SparsePoseGraph::Constraint2D> > const&) ()
#7  0x000000000060639a in cartographer::mapping_2d::sparse_pose_graph::ConstraintBuilder::FinishComputation(int) ()
#8  0x000000000066351d in cartographer::common::ThreadPool::DoWork() ()
#9  0x00007ffff5186c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x00007ffff6d6a6fa in start_thread (arg=0x7fffdf7fe700) at pthread_create.c:333
#11 0x00007ffff4bf5b5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

I should add that the segfault also disappears while running under ASan, but not TSan or GDB.

I got the segfault to happen in valgrind
Here is the output:
==15634== Memcheck, a memory error detector
==15634== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==15634== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==15634== Command: /home/rohan/cartographer_catkin_ws/devel_isolated/cartographer_ros/lib/cartographer_ros/cartographer_node __name:=cartographer __log:=/home/rohan/.ros/log/d8b26ef8-5b38-11e6-863d-801f0286f30c/cartographer-1.log
==15634==
[ INFO] [1470423499.181136001]: I0805 11:58:19.000000 15634 submaps.cc:175] Added submap 1
[ INFO] [1470423500.068260510]: I0805 11:58:20.000000 15634 motion_filter.cc:42] Motion filter reduced the number of scans to -nan%.
[ INFO] [1470423561.852290134]: I0805 11:59:21.000000 15634 motion_filter.cc:42] Motion filter reduced the number of scans to 5.4%.
==15634== Thread 8:
==15634== Invalid read of size 8
==15634==    at 0x4F7BFCF: ceres::internal::ReorderProgramForSparseNormalCholesky(ceres::SparseLinearAlgebraLibraryType, ceres::OrderedGroups<double*> const&, ceres::internal::Program*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) (in /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1.11.0)
==15634==    by 0x4F9B2DC: ceres::internal::TrustRegionPreprocessor::Preprocess(ceres::Solver::Options const&, ceres::internal::ProblemImpl*, ceres::internal::PreprocessedProblem*) (in /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1.11.0)
==15634==    by 0x4F933F3: ceres::Solver::Solve(ceres::Solver::Options const&, ceres::Problem*, ceres::Solver::Summary*) (in /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1.11.0)
==15634==    by 0x4F93EDE: ceres::Solve(ceres::Solver::Options const&, ceres::Problem*, ceres::Solver::Summary*) (in /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1.11.0)
==15634==    by 0x601504: cartographer::mapping_2d::sparse_pose_graph::OptimizationProblem::Solve(std::vector<cartographer::mapping::SparsePoseGraph::Constraint2D, std::allocator<cartographer::mapping::SparsePoseGraph::Constraint2D> > const&, std::vector<cartographer::mapping::Submaps const*, std::allocator<cartographer::mapping::Submaps const*> > const&, std::vector<cartographer::transform::Rigid2<double>, std::allocator<cartographer::transform::Rigid2<double> > > const&, std::vector<cartographer::transform::Rigid2<double>, std::allocator<cartographer::transform::Rigid2<double> > >*, std::vector<cartographer::transform::Rigid2<double>, std::allocator<cartographer::transform::Rigid2<double> > >*) (in /home/rohan/cartographer_catkin_ws/devel_isolated/cartographer_ros/lib/cartographer_ros/cartographer_node)
==15634==    by 0x5F69A8: cartographer::mapping_2d::SparsePoseGraph::RunOptimization() (in /home/rohan/cartographer_catkin_ws/devel_isolated/cartographer_ros/lib/cartographer_ros/cartographer_node)
==15634==    by 0x5F752C: std::_Function_handler<void (std::vector<cartographer::mapping::SparsePoseGraph::Constraint2D, std::allocator<cartographer::mapping::SparsePoseGraph::Constraint2D> > const&), cartographer::mapping_2d::SparsePoseGraph::HandleScanQueue()::{lambda(std::vector<cartographer::mapping::SparsePoseGraph::Constraint2D, std::allocator<cartographer::mapping::SparsePoseGraph::Constraint2D> > const&)#1}>::_M_invoke(std::_Any_data const&, std::vector<cartographer::mapping::SparsePoseGraph::Constraint2D, std::allocator<cartographer::mapping::SparsePoseGraph::Constraint2D> > const&) (in /home/rohan/cartographer_catkin_ws/devel_isolated/cartographer_ros/lib/cartographer_ros/cartographer_node)
==15634==    by 0x606399: cartographer::mapping_2d::sparse_pose_graph::ConstraintBuilder::FinishComputation(int) (in /home/rohan/cartographer_catkin_ws/devel_isolated/cartographer_ros/lib/cartographer_ros/cartographer_node)
==15634==    by 0x66351C: cartographer::common::ThreadPool::DoWork() (in /home/rohan/cartographer_catkin_ws/devel_isolated/cartographer_ros/lib/cartographer_ros/cartographer_node)
==15634==    by 0x7879C7F: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==15634==    by 0x5C986F9: start_thread (pthread_create.c:333)
==15634==  Address 0x8 is not stack'd, malloc'd or (recently) free'd
==15634==
==15634==
==15634== Process terminating with default action of signal 11 (SIGSEGV)
==15634==  Access not within mapped region at address 0x8
==15634==    at 0x4F7BFCF: ceres::internal::ReorderProgramForSparseNormalCholesky(ceres::SparseLinearAlgebraLibraryType, ceres::OrderedGroups<double*> const&, ceres::internal::Program*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) (in /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1.11.0)
==15634==    by 0x4F9B2DC: ceres::internal::TrustRegionPreprocessor::Preprocess(ceres::Solver::Options const&, ceres::internal::ProblemImpl*, ceres::internal::PreprocessedProblem*) (in /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1.11.0)
==15634==    by 0x4F933F3: ceres::Solver::Solve(ceres::Solver::Options const&, ceres::Problem*, ceres::Solver::Summary*) (in /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1.11.0)
==15634==    by 0x4F93EDE: ceres::Solve(ceres::Solver::Options const&, ceres::Problem*, ceres::Solver::Summary*) (in /home/rohan/cartographer_catkin_ws/devel_isolated/ceres_catkin/lib/libceres.so.1.11.0)
==15634==    by 0x601504: cartographer::mapping_2d::sparse_pose_graph::OptimizationProblem::Solve(std::vector<cartographer::mapping::SparsePoseGraph::Constraint2D, std::allocator<cartographer::mapping::SparsePoseGraph::Constraint2D> > const&, std::vector<cartographer::mapping::Submaps const*, std::allocator<cartographer::mapping::Submaps const*> > const&, std::vector<cartographer::transform::Rigid2<double>, std::allocator<cartographer::transform::Rigid2<double> > > const&, std::vector<cartographer::transform::Rigid2<double>, std::allocator<cartographer::transform::Rigid2<double> > >*, std::vector<cartographer::transform::Rigid2<double>, std::allocator<cartographer::transform::Rigid2<double> > >*) (in /home/rohan/cartographer_catkin_ws/devel_isolated/cartographer_ros/lib/cartographer_ros/cartographer_node)
==15634==    by 0x5F69A8: cartographer::mapping_2d::SparsePoseGraph::RunOptimization() (in /home/rohan/cartographer_catkin_ws/devel_isolated/cartographer_ros/lib/cartographer_ros/cartographer_node)
==15634==    by 0x5F752C: std::_Function_handler<void (std::vector<cartographer::mapping::SparsePoseGraph::Constraint2D, std::allocator<cartographer::mapping::SparsePoseGraph::Constraint2D> > const&), cartographer::mapping_2d::SparsePoseGraph::HandleScanQueue()::{lambda(std::vector<cartographer::mapping::SparsePoseGraph::Constraint2D, std::allocator<cartographer::mapping::SparsePoseGraph::Constraint2D> > const&)#1}>::_M_invoke(std::_Any_data const&, std::vector<cartographer::mapping::SparsePoseGraph::Constraint2D, std::allocator<cartographer::mapping::SparsePoseGraph::Constraint2D> > const&) (in /home/rohan/cartographer_catkin_ws/devel_isolated/cartographer_ros/lib/cartographer_ros/cartographer_node)
==15634==    by 0x606399: cartographer::mapping_2d::sparse_pose_graph::ConstraintBuilder::FinishComputation(int) (in /home/rohan/cartographer_catkin_ws/devel_isolated/cartographer_ros/lib/cartographer_ros/cartographer_node)
==15634==    by 0x66351C: cartographer::common::ThreadPool::DoWork() (in /home/rohan/cartographer_catkin_ws/devel_isolated/cartographer_ros/lib/cartographer_ros/cartographer_node)
==15634==    by 0x7879C7F: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==15634==    by 0x5C986F9: start_thread (pthread_create.c:333)
==15634==    by 0x7E5FB5C: clone (clone.S:109)
==15634==  If you believe this happened as a result of a stack
==15634==  overflow in your program's main thread (unlikely but
==15634==  possible), you can try to increase the size of the
==15634==  main thread stack using the --main-stacksize= flag.
==15634==  The main thread stack size used in this run was 8388608.
==15634==
==15634== HEAP SUMMARY:
==15634==     in use at exit: 131,395,694 bytes in 23,975 blocks
==15634==   total heap usage: 7,911,062 allocs, 7,887,087 frees, 1,637,650,075 bytes allocated
==15634==
==15634== LEAK SUMMARY:
==15634==    definitely lost: 184 bytes in 1 blocks
==15634==    indirectly lost: 0 bytes in 0 blocks
==15634==      possibly lost: 4,800 bytes in 15 blocks
==15634==    still reachable: 131,390,710 bytes in 23,959 blocks
==15634==         suppressed: 0 bytes in 0 blocks
==15634== Rerun with --leak-check=full to see details of leaked memory
==15634==
==15634== For counts of detected and suppressed errors, rerun with: -v
==15634== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

Here is a bag file that reproduces the error:
https://drive.google.com/file/d/0B2W9KeFIetG2NW9mNm5relQ0WEU/view?usp=sharing

Hi, I looked into it a bit and managed to reproduce the issue. I used the "Standalone Workspace" instructions for building on xenial/kinetic.
On my trusty/indigo system these instructions work without the issue. Likewise, installing as described in https://github.com/googlecartographer/cartographer, and using catkin only for cartographer and cartographer_ros (and not ceres_catkin) also works on xenial/kinetic. So this could at least be a short term workaround.
All crashes so far seem to be Ceres related, and can even be reproduced by some of the unit tests. A pretty small test which crashes on my system is https://github.com/googlecartographer/cartographer/blob/master/cartographer/mapping_2d/scan_matching/ceres_scan_matcher_test.cc.

@wohe could you share a smaller reproduction with me if possible? and i will work on fixing it?

This might be a bug in ceres_catkin, here it is checking out a commit (6a13e39) from the ceres repo, and saying that it is v1.11, https://github.com/ethz-asl/ceres_catkin/blob/master/CMakeLists.txt#L12.
In the github repo,  v1.11 has is tagged as commit 4145556.
https://github.com/ceres-solver/ceres-solver/releases/tag/1.11.0

Already checked that. The hash is referring to upstream at https://ceres-solver.googlesource.com/ceres-solver/ and is indeed 1.11. Newer version and removing patches also did not help.

I investigated a bit more, and found the following:
The two functions from Ceres that lead to crashes (ComputeCovarianceValuesUsingSuiteSparseQR and OrderingForSparseNormalCholeskyUsingSuiteSparse) both have UsingSuiteSparse in their name, and it turns out that the version of catkin suitesparse and trusty's SuiteSparse are both 4.2.1, but xenial has 4.4.6. I uninstalled the xenial libsuitesparse-dev (which should be entirely unused in this build) and the stack smashing bug went away.
My conclusion: The catkin builds of Ceres and SuiteSparse are not ignoring the system's SuiteSparse as they should and compile/link together parts belonging to different versions of SuiteSparse. I don't know enough of catkin to fix this easily. Any ideas?
It seemed like it could be an ideal solution on xenial to remove the catkin build of Ceres, and just use xenial's libceres-dev. Unfortunately there is https://bugs.launchpad.net/ubuntu/+source/ceres-solver/+bug/1596296.

We can add FindCeres.cmake as part of ROS's cmake_modules (https://github.com/ros/cmake_modules) and add that as a dependency.

I think I found a solution that allows us to fix it on our side for now. @rohbotics, can you verify that #18 solves the issue for you? I tried it here with trusty/indigo and xenial/kinetic. Both seem fine.

#18 fixes this issue

Hi, I'm having the stack smashing as well.
roslaunch cartographer_ros demo_backpack_2d.launch bag_filename:=${HOME}/Downloads/cartographer_paper_deutsches_museum.bag
... logging to /home/bo/.ros/log/aa0e2930-07d5-11e7-a596-e4a7a035a11f/roslaunch-bo-ThinkPad-T460p-10557.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://bo-ThinkPad-T460p:34053/

SUMMARY
========

PARAMETERS
 * /robot_description: <...>
 * /rosdistro: kinetic
 * /rosversion: 1.12.7
 * /use_sim_time: True

NODES
  /
    cartographer_node (cartographer_ros/cartographer_node)
    playbag (rosbag/play)
    robot_state_publisher (robot_state_publisher/robot_state_publisher)
    rviz (rviz/rviz)

auto-starting new master
process[master]: started with pid [10570]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to aa0e2930-07d5-11e7-a596-e4a7a035a11f
process[rosout-1]: started with pid [10583]
started core service [/rosout]
process[robot_state_publisher-2]: started with pid [10590]
process[cartographer_node-3]: started with pid [10602]
process[rviz-4]: started with pid [10610]
process[playbag-5]: started with pid [10618]
[ INFO] [1489399992.671926019]: I0313 11:13:12.000000 10602 configuration_file_resolver.cc:40] Found '/home/bo/catkin_ws/install_isolated/share/cartographer_ros/configuration_files/backpack_2d.lua' for 'backpack_2d.lua'.
[ INFO] [1489399992.672306517]: I0313 11:13:12.000000 10602 configuration_file_resolver.cc:40] Found '/home/bo/catkin_ws/install_isolated/share/cartographer/configuration_files/map_builder.lua' for 'map_builder.lua'.
[ INFO] [1489399992.672354654]: I0313 11:13:12.000000 10602 configuration_file_resolver.cc:40] Found '/home/bo/catkin_ws/install_isolated/share/cartographer/configuration_files/map_builder.lua' for 'map_builder.lua'.
[ INFO] [1489399992.672440033]: I0313 11:13:12.000000 10602 configuration_file_resolver.cc:40] Found '/home/bo/catkin_ws/install_isolated/share/cartographer/configuration_files/trajectory_builder_2d.lua' for 'trajectory_builder_2d.lua'.
[ INFO] [1489399992.672481481]: I0313 11:13:12.000000 10602 configuration_file_resolver.cc:40] Found '/home/bo/catkin_ws/install_isolated/share/cartographer/configuration_files/trajectory_builder_2d.lua' for 'trajectory_builder_2d.lua'.
[ INFO] [1489399992.672639777]: I0313 11:13:12.000000 10602 configuration_file_resolver.cc:40] Found '/home/bo/catkin_ws/install_isolated/share/cartographer/configuration_files/trajectory_builder_3d.lua' for 'trajectory_builder_3d.lua'.
[ INFO] [1489399992.672681100]: I0313 11:13:12.000000 10602 configuration_file_resolver.cc:40] Found '/home/bo/catkin_ws/install_isolated/share/cartographer/configuration_files/trajectory_builder_3d.lua' for 'trajectory_builder_3d.lua'.
[ INFO] [1489399992.672858258]: I0313 11:13:12.000000 10602 configuration_file_resolver.cc:40] Found '/home/bo/catkin_ws/install_isolated/share/cartographer/configuration_files/sparse_pose_graph.lua' for 'sparse_pose_graph.lua'.
[ INFO] [1489399992.672896972]: I0313 11:13:12.000000 10602 configuration_file_resolver.cc:40] Found '/home/bo/catkin_ws/install_isolated/share/cartographer/configuration_files/sparse_pose_graph.lua' for 'sparse_pose_graph.lua'.
[ INFO] [1489399992.792620165]: I0313 11:13:12.000000 10602 submaps.cc:176] Added submap 1
[ INFO] [1489399992.792723199]: I0313 11:13:12.000000 10602 map_builder_bridge.cc:37] Added trajectory with ID '0'.
[ INFO] [1489399993.392465301,<PHONE_NUMBER>.509920142]: I0313 11:13:13.000000 10602 ordered_multi_queue.cc:172] All sensor data for trajectory 0 is available starting at '635682438164885689'.
*** stack smashing detected ***: /home/bo/catkin_ws/install_isolated/lib/cartographer_ros/cartographer_node terminated
[cartographer_node-3] process has died [pid 10602, exit code -6, cmd /home/bo/catkin_ws/install_isolated/lib/cartographer_ros/cartographer_node -configuration_directory /home/bo/catkin_ws/install_isolated/share/cartographer_ros/configuration_files -configuration_basename backpack_2d.lua echoes:=horizontal_laser_2d __name:=cartographer_node __log:=/home/bo/.ros/log/aa0e2930-07d5-11e7-a596-e4a7a035a11f/cartographer_node-3.log].
log file: /home/bo/.ros/log/aa0e2930-07d5-11e7-a596-e4a7a035a11f/cartographer_node-3*.log
0x1b98050 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xf95250) ): Attempt to set a screen on a child window.
0x1b98470 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xf95250) ): Attempt to set a screen on a child window.
0x1b8c830 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xf95250) ): Attempt to set a screen on a child window.
0x1b8b540 void QWindowPrivate::setTopLevelScreen(QScreen*, bool) ( QScreen(0xf95250) ): Attempt to set a screen on a child window.
^C[playbag-5] killing on exit
[rviz-4] killing on exit
[robot_state_publisher-2] killing on exit
[rosout-1] killing on exit
[master] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done

I saw that you changed to a different ceres package now. I have the OpenBLAS and Suitesparse libraries installed on my PC, do I need to uninstall them?

I found the error, the suitsparse library was still in the catkin workspace devel folder. I had to remove it from there as well and it works now.

You can modify the CMakeLists.txt to not use SuitSparse (and use EigenSparse instead). It appears that there is a bug in the SuiteSparse (v4.4.6) that comes on Ubuntu 16.04. It wasn't obvious how to compile the newer SuiteSparse on Ubuntu and get it installed into the right folders.

Also ran into the stack smashing issue, similar to what is described in the OP. In my case it was caused by another repo containing a suitesparse package (https://github.com/ethz-asl/kalibr/tree/master/suitesparse). After removing that and doing a clean build, the problem is gone. Adding this here in the interest of people googling for stack smashing cartographer who probably end up reading this issue :)


Update pages to show latest release

cherry-picked re-generated pages from 1.x rev. 29870448fb6bb7afd08503545543557224fa1310 to make sure people are using 1.5 release

Please, complete the changes by adding yourself as a contributor in the POM.

Done


Memory leak in SimpleProvisionMetrics

SimpleProvisionMetrics should use a map with thread id instead of ThreadLocal to avoid memory leaks from not cleaning up ThreadLocals

3.x branch no longer under development


Fold flow.tensor.reshape & other ops into dispatch regions when possible (for sparse)

The ConvertToFlowTensorOps.cpp pass is inserting flow.tensor.reshape and other ops prior to dispatch region formation and dispatch region formation does not pull them in. This works today because these ops acting on dense tensors are metadata-only or DMA-compatible but in the case of sparse tensors (or tensors with any other weird layouts) this won't always be the case. Any op that may need to be performed via dispatch - like reshaping/slicing sparse tensors - we'll want to be fused into other dispatches similar to how we want padding/packing fused into dispatches.
Reshapes in the sparse case should just be a reordering and not change the underlying storage size of the tensor. Slices are different in that they will but if we think of fusing them into producers we can either gain efficiency savings (doing less work) or do the packing in a single location. For example:
%0 = flow.dispatch @foo(...) -> tensor<2x10xf32>
%1 = flow.tensor.slice %0[%c0, %c0 for %c1, %c10] : tensor<2x10xf32> -> tensor<1x10xf32>
%2 = flow.tensor.slice %0[%c1, %c0 for %c1, %c10] : tensor<2x10xf32> -> tensor<1x10xf32>

if %0, %1, or %2 are sparse then this would have to be done as another dispatch or ideally be folded into @foo:
%0#2 = flow.dispatch @foo(...) -> (tensor<1x10xf32>, tensor<1x10xf32>)

(reshape factors in here too because they tend to crop up around the slices)
This may be worth doing in the dense case as well (and would keep our code paths easier to understand).
Inserts (flow.tensor.update) are also going to need to be folded in to dispatches on sparse things as the entire tensor needs to be repacked. I suspect in that case we may want a dedicated dispatch or fold into consumers: the entire post-update tensor needs to be repacked. If the consumers are elementwise/etc then fusing those is cheap and amortizes the cost of the repacking. We still want these as standalone ops for the dense cases as they can be used for placement allocations in that case, while in the sparse case the target would be tied operands and maybe the operation could happen in-place.
Requires some more experimentation with sparse stuff to really know what to do. Fun fun!

Some of this presumed that we don't want the sparse packing to be standalone - it may be the case that non-elementwise producers should never try to pack or partially pack just local tiles and have a repacking dispatch that does the whole output. Of course, if to start we are only using sparse tensors for large constant/infrequently changing tensors then we may be able to ignore all of this :)

Probably obsolete.